Method for computing all occurrences of a compound event from occurrences of primitive events

ABSTRACT

A method for computing all occurrences of a compound event from occurrences of primitive events where the compound event is a defined combination of the primitive events. The method includes the steps of: (a) defining primitive event types; (b) defining combinations of the primitive event types as a compound event type; (c) inputting the primitive event occurrences, such occurrences being specified as the set of temporal intervals over which a given primitive event type is true; and (d) computing the compound event occurrences, such occurrences being specified as the set of temporal intervals over which the compound event type is true, where the sets of temporal intervals in steps (c) and (d) are specified as smaller sets of spanning intervals, each spanning interval representing a set of intervals.

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. ProvisionalApplication No. 60/247,474 filed Nov. 10, 2000.

FIELD OF THE INVENTION

[0002] The present invention relates generally to methods for computingoccurrences of compound events and, more particularly, to methods forcomputing occurrences of compound events from occurrences of primitiveevents where such occurrences are specified as a set of temporalintervals over which the compound event type is true.

BACKGROUND OF THE INVENTION

[0003] Event logic provides a calculus for forming compound event typesas expressions over primitive event types. The syntax and semantics ofevent logic will be described momentarily. Event-logic expressionsdenote event types, not event occurrences. As such, they do not havetruth values. Rather, they are predicates that describe the truthconditions that must hold of an interval for an event to occur. Incontrast, an event-occurrence formula does have a truth value. If Φ isan event-logic expression that denotes a primitive or compound eventtype, and i is an interval, then Φ@i is an atomic event-occurrenceformula that is true if and only if the truth conditions for the eventtype Φ hold of the interval i.

[0004] Φ@i denotes coincidental occurrence, the fact that an occurrenceof Φ started at the beginning of i and finished at the end of i. Φ@iwould not hold if an occurrence of Φ did not precisely coincide with i,but instead overlapped with i. Event types have internal temporalstructure that render this distinction important. In the case ofprimitive event types, that structure is simple. Each primitive eventtype is derived from a static predicate. A primitive event type Φ holdsof an interval if the corresponding static predicate φ holds of everyinstant in that interval. This means that

({overscore (φ)}@i) and {overscore (

φ)}@i might have different truth values. For example, if φ is true ofevery instant in [0, 2) and false of every other instant, then

({overscore (φ)}@[1, 3)) is true while {overscore (

φ)}@[1, 3) is false. Event logic takes coincidental occurrence to be aprimitive notion. As will be demonstrated below, overlapping occurrenceis a derived notion that can be expressed in terms of coincidentaloccurrence using compound event-logic expressions.

[0005] Two auxiliary notions are needed to define the syntax andsemantics of event logic. First, there are thirteen possible relationsbetween two intervals. These relations are denoted as =, <, >, m, mi, o,oi, s, si, f, fi, d, and di and referred to collectively as Allenrelations throughout this disclosure. Second, the span of two intervalsi and j, denoted SPAN (i, j), is defined as the smallest super-intervalof both i and j.

[0006] The syntax of event logic is defined as follows. We are givenfinite disjoint sets of constant and variable symbols along with afinite set of primitive event-type symbols, each of a specified arity.Constant symbols, such as red-block and hand, denote objects whileprimitive event-type symbols, such as SUPPORTS, denote parameterizedprimitive event types. An atomic event-logic expression is a primitiveevent-type symbol of arity n applied to a sequence of n constants orvariables. For example, SUPPORTS (green-block, x). An event-logicexpression is either an atomic event-logic expression or one of thecompound event-logic expressions

Φ, Φ,

Ψ, ∀xΦ, ∃xΦ, Φ

_(R) Ψ, or ⋄_(R)Φ, where Φ and Ψ are event-logic expressions, x is avariable, and

R⊂ {=, <, >, m, mi, o, oi, s, si, f, fi, d, di}.

[0007] Informally, the semantics of compound event-logic expressions isdefined as follows:

[0008]

Φ denotes the non-occurrence of Φ. An occurrence of

Φ coincides with i if no occurrence of Φ coincides with i. Note that (

Φ)@i could be true, even if an occurrence of Φ overlapped with i, solong as no occurrence of Φ coincided with i.

[0009] ΦΨ denotes the occurrence of either Φ or Ψ.

[0010] ∀xΦ denotes the simultaneous occurrence of Φ for all objects.

[0011] ∃xΦ denotes the occurrence of Φ for some object.

[0012] Φ

_(R) Ψ denotes the occurrence of both Φ and Ψ. The occurrences of Φ andΨ need not be simultaneous. The subscript R specifies a set of allowedAllen relations between the occurrences of Φ and Ψ. If occurrences of Φand Ψ coincide with i and j respectively, and i r j for some rεR ,thenan occurrence of Φ

_(R) Ψ coincides with the span of i and j. The special case Φ

_({=}) Ψ is abbreviated simply as Φ

Ψ without any subscript. Φ

Ψ describes an aggregate event where both Φ and ⊥ occur simultaneously.The special case Φ

_({m})Ψ is also abbreviated as (Φ;Ψ. Φ;Ψ describes an aggregate eventwhere an occurrence of Φ is immediately followed by an occurrence of Ψ.

[0013] An occurrence of ⋄_(R)Φ coinciding with i denotes an occurrenceof Φ at some other interval j such that j r i for some r εR. ⋄_(R) canact as a tense operator. Expressions such as ⋄_({<})Φ, ⋄_({>}Φ, ⋄)_({m})Φ, and ⋄_({mi})Φ specify that Φ happened in the noncontiguouspast, noncontiguous future, contiguous past, or contiguous futurerespectively. The ⋄_(R) operator can also be used to derive overlappedoccurrence from coincidental occurrence. An occurrence of⋄_({=, o, oi, s, si, f, fi, d, di})Φ coincides with i if an occurrenceof Φ overlaps with i. I abbreviate ⋄_({=, o, oi, s, si, f, fi, d, di})Φsimply as ⋄Φ without any subscript. Note that while (

Φ)@i indicates that no occurrence of Φ coincided with i, (

⋄Φ)@i indicates that no occurrence of Φ overlapped with i.

[0014] Formally, the truth of an atomic event-occurrence formula Φ@i isdefined relative to a model. Let I be the set of all intervals. A modelM is a triple <O, T, P>, where 0 is a set of objects, T is a map fromconstants and variables to objects from O, and P is map from primitiveevent-type symbols of arity n to subsets of$I \times {\underset{\underset{n}{}}{O \times \ldots \times O}.}$

[0015] P thus maps primitive event-type symbols to relations that takean interval as their first argument, in addition to the remaining objectparameters. T [x:=o] denotes a map that is identical to T except that itmaps the variable x to the object o. The semantics of event logic isformally defined by specifying an entailment relation M

Φ@i as follows:

[0016] <O, T, P>

p(t₁, . . . ,t_(n))@i if and only if <i, T(t₁), . . . ,T(t_(n))>εP(p).

[0017] M

(

Φ)@i if and only if M|≠Φ@i.

[0018] M

(Φ

Ψ)@i if and only if M

Φ@i or M

Ψ@i.

[0019] <O, T, P>

∀xΦ)@i if and only if <O, T[x:=o], P>

Φ@i for ever y o εO.

[0020] <O, T, P>

(∃xΦ)@i if and only if <O, T[x:=o], P>

Φ@i for some oεO.

[0021] M

(Φ

_(R)Ψ)@i if and only if there exist two intervals j and k such thati=SPAN j, k), j r k for some rεR, M

Φ@j, and M

Ψ@k.

[0022] M

(⋄_(R)Φ)@i if and only if there exists some interval j such that j r ifor some rεR and M

Φ@j.

[0023] The overall goal of the event-classification component is toinfer all occurrences of a given set of compound event types from agiven set of primitive event occurrences. Let us define ε(M, Φ) to be{i|M

Φ@i}. In principle, ε(M, Φ) could by implemented as a straightforwardapplication of the formal semantics for event logic as specified above.There is a difficulty in doing so, however. Primitive event types oftenhave the property that they are liquid. Liquid events have the followingtwo properties. First, if they are true during an interval i, then theyare also true during any subinterval of i. Second, if they are trueduring two overlapping intervals i and j, then they are also true duringSPAN(i, j). When primitive event types are liquid, they will hold overan infinite number of subintervals. This renders the formal semanticsinappropriate for a computational implementation. Even if one limitsoneself to intervals with integral endpoints, liquid primitive eventtypes will hold over quadratically many subintervals of the scenesequence. And a straightforward computational implementation of theformal semantics would be inefficient. A central result of thisdisclosure is a novel representation, called spanning intervals, thatallows an efficient representation of the infinite sets of subintervalsover which liquid event types hold along with an efficient inferenceprocedure that operates on that representation. This representation, andthe inference procedure that implements ε(M, Φ), are presented below.

SUMMARY OF THE INVENTION

[0024] Therefore it is an object of the present invention to provide amethod for computing all occurrences of a compound event fromoccurrences of primitive events which overcomes the problems associatedwith the prior art methods.

[0025] Accordingly, a method for computing all occurrences of a compoundevent from occurrences of primitive events is provided where thecompound event is a defined combination of the primitive events. Themethod comprises the steps of: (a) defining primitive event types; (b)defining combinations of the primitive event types as a compound eventtype; (c) inputting the primitive event occurrences, such occurrencesbeing specified as the set of temporal intervals over which a givenprimitive event type is true; and (d) computing the compound eventoccurrences, such occurrences being specified as the set of temporalintervals over which the compound event type is true, where the set oftemporal intervals in steps (c) and (d) are specified as smaller sets ofspanning intervals, each spanning interval representing a set ofintervals. Preferably, the spanning intervals take the form _(α)[_(γ)[i,j]_(δ), _(ε)[k, l]_(ζ)]_(β), where α, β, γ, δ, ε, and ζ are Booleanvalues, i, j, k, and l are real numbers, _(α)[_(γ)[i, j]_(δ), _(ε[k, l])_(ζ)]_(β) represents the set of all intervals _(α)[p, q]_(β), wherei≦_(γ) p≦_(δ) j and k≦_(ε) q≦_(ζ) l, _(α)[p, q]_(β) represents the setof all points r, where p≦_(α) r≦_(β) q, and x≦_(θ) y means x≦y when θ istrue and x<y when θ is false.

[0026] The methods of the present invention provide an efficientimplementation of ε(M, Φ) along with six subroutines used by ε(M, Φ),namely <i>, i₁ ∩ i₂,

i, SPAN (i₁, i₂),

(r, i), and

(i, r, j).

[0027] Also provided are a computer program product for carrying out themethods of the present invention and a program storage device for thestorage of the computer program product therein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0028] These and other features, aspects, and advantages of the methodsof the present invention will become better understood with regard tothe following description, appended claims, and accompanying drawingswhere:

[0029]FIG. 1 illustrates a flowchart of a preferred implementation ofthe methods steps of the present invention;

[0030]FIG. 2 illustrates a flowchart of the structural induction processused to implement step 108 of FIG. 1;

[0031]FIG. 3 illustrates the primitive event types used by thecomputer-system implementation of one application of the methods of thepresent invention as discussed in the “Example” section below;

[0032]FIG. 4 illustrates the lexicon of compound event types used by thecomputer-system implementation of one application of the methods of thepresent invention as discussed in the “Example” section below;

[0033]FIGS. 5A and 5B illustrate sequences of video frames depicting apick up and put down event respectively, wherein the results ofperforming segmentation, tracking, and model reconstruction have beenoverlayed on the video frames;

[0034]FIGS. 6A and 6B illustrate the output of the event-classificationmethods of the present invention applied to the model sequences fromFIGS. 5A and 5B, respectively;

[0035]FIGS. 7A, 7B, 7C, 7D, and 7E illustrate sequences of video framesdepicting stack, unstack, move, assemble and disassemble events, whereinthe results of performing segmentation, tracking, and modelreconstruction have been overlayed on the video frames;

[0036]FIGS. 8A, 8B, 8C, 8D, and 8E illustrate the output of theevent-classification methods of the present invention applied to themodel sequences from FIGS. 7A, 7B, 7C, 7D, and 7E, respectively;

[0037]FIGS. 9A, 9B, 9C, and 9D illustrate sequences of video framesdepicting: a pick up event from the left instead of from the right; apick up event with extraneous objects in the field of view; a sequenceof a pick up event followed by a put down event followed by another pickup event followed by another put down event; and two simultaneous pickup events, respectively, wherein the results of performing segmentation,tracking, and model reconstruction have been overlayed on the videoframes;

[0038]FIGS. 10A, 10B, 10C, and 10D illustrate the output of theevent-classification methods of the present invention applied to themodel sequences from FIGS. 9A, 9B, 9C, and 9D, respectively;

[0039]FIGS. 11A and 11B illustrate sequences of video frames depictingnon-events, wherein the results of performing segmentation, tracking,and model reconstruction have been overlayed on the video frames; and

[0040]FIGS. 12A and 12B illustrate the output of theevent-classification methods of the present invention applied to themodel sequences from FIGS. 11A and 11B, respectively.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0041] Referring now to FIG. 1, there is illustrated a flowchartpresenting a general overview of the method steps for computing alloccurrences of a compound event from occurrences of primitive events,where the compound event is a defined combination of the primitiveevents. The method comprises defining primitive event types at step 102.At step 104 combinations of the primitive event types are defined as acompound event type. At step 106, the primitive event occurrences areinput, such occurrences being specified as the set of temporal intervalsover which a given primitive event type is true. Lastly, at step 108,the compound event occurrences are computed. Such occurrences beingspecified as the set of temporal intervals over which the compound eventtype is true, where, the set of temporal intervals in steps 106 and 108are specified as smaller sets of spanning intervals, each spanninginterval representing a set of intervals.

[0042] A detailed explanation of the general method steps in FIG. 1 willnow be discussed.

[0043] Intervals

[0044] One might try to implement event logic using only closedintervals of the form [q, r], where q≦r. Such a closed interval wouldrepresent the set {p|q≦p≦r} of real numbers. With such closed intervals,one would define the Allen relations as follows: $\begin{matrix}{\left\lbrack {q_{1},r_{1}} \right\rbrack = \left\lbrack {q_{2},r_{2}} \right\rbrack} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} = {{q_{2}\bigwedge r_{1}} = r_{2}}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack < \left\lbrack {q_{2},r_{2}} \right\rbrack} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {r_{1} < q_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack > \left\lbrack {q_{2},r_{2}} \right\rbrack} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} > r_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack {m\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {r_{1} = q_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack m\quad {i\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} = r_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack {o\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} < q_{2} < r_{1} < r_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack o\quad {i\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {r_{1} > r_{2} > q_{1} > q_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack {s\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} = {{q_{2}\bigwedge r_{1}} < r_{2}}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack s\quad {i\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} = {{q_{2}\bigwedge r_{1}} > r_{2}}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack {f\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {{q_{1} > {q_{2}\bigwedge r_{1}}} = r_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack {{fi}\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {{q_{1} < {q_{2}\bigwedge r_{1}}} = r_{2}}} \\{\left\lbrack {q_{1},r_{1}} \right\rbrack {d\left\lbrack {q_{2},r_{2}} \right\rbrack}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} > {q_{2}\bigwedge r_{1}} < r_{2}}} \\{\quad {\left\lbrack {q_{1},r_{1}} \right\rbrack {{di}\left\lbrack {q_{2},r_{2}} \right\rbrack}}} & \underset{\underset{\_}{\_}}{\Delta} & {\quad {q_{1} < {q_{2}\bigwedge r_{1}} > r_{2}}}\end{matrix}$

[0045] One difficulty with doing so is that it would possible for morethan one Allen relation to hold between two intervals when one or bothof them are instantaneous intervals, such as [q, q]. Both m and s andwould hold between [q, q] and [q, r], both mi and si would hold between[q, r] and [q, q], both m and fi would hold between [q, r] and [r, r],both mi and f would hold between [r, r] and [q, r], and =, m, and miwould all hold between [q, q] and itself. To create a domain whereexactly one Allen relation holds between any pair of intervals, let usconsider both open and closed intervals. The intervals (q, r], [q, r),and (q, r), where q<r, represent the sets {p|q<p≦r}, {p|q≦p<r} and{p|q<p<r} of real numbers respectively. The various kinds of open andclosed intervals can be unified into a single representation _(α)[q,r]_(β), where α and β are true or false to indicate the interval beingclosed or open on the left or right respectively. To do this, let us useq≦_(α)r to mean q≦r when α is true and q<r when α is false. Similarly,let us use q≧_(α)r to mean q≧r when α is true and q>r when α is false.With these, _(α)[q, r]_(β) represents the set {p|q≦_(α)p≦_(β)r} of realnumbers. Given this, one can define the Allen relations as follows:$\begin{matrix}\begin{matrix}{{\quad_{\alpha_{1}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}} =_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack_{\beta_{2}}} & \underset{\underset{\_}{\_}}{\quad \Delta} & {\quad {q_{1} = {{q_{2}\bigwedge\alpha_{1}} = {{\alpha_{2}\bigwedge r_{1}} = {{r_{2}\bigwedge\beta_{1}} = \beta_{2}}}}}}\end{matrix} & (1) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}} <_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {r_{1} \leq_{{\beta_{1}\bigwedge{\alpha_{2}}}}q_{2}}}\end{matrix} & (2) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}} >_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack_{\beta_{2}}} & \underset{\underset{\_}{\_}}{\quad \Delta} & {\quad {q_{1} \geq_{{\alpha_{1}\bigwedge{\beta_{2}}}}r_{2}}}\end{matrix} & (3) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}{m_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {r_{1} = {{q_{2}\bigwedge\beta_{1}} \neq \alpha_{2}}}}\end{matrix} & (4) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}m\quad {i_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {q_{1} = {{r_{2}\bigwedge\alpha_{1}} \neq \beta_{2}}}}\end{matrix} & (5) \\\begin{matrix}{{\quad_{\quad \alpha_{1}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}{o_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {q_{1} \leq_{\alpha_{1}\bigwedge{\alpha_{2}}}q_{2} \leq_{\beta_{1}\bigwedge\alpha_{2}}r_{1} \leq_{{\beta_{1}\bigwedge\beta_{2}}}r_{2}}}\end{matrix} & (6) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}o\quad {i_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {r_{1} \geq_{\beta_{1}\bigwedge{\beta_{2}}}r_{2} \geq_{\alpha_{1}\bigwedge\beta_{2}}q_{1} \geq_{{\alpha_{1}\bigwedge\alpha_{2}}}q_{2}}}\end{matrix} & (7) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}{s_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {q_{1} = {{q_{2}\bigwedge\alpha_{1}} = {{\alpha_{2}\bigwedge r_{1}} \leq_{{\beta_{1}\bigwedge\beta_{2}}}r_{2}}}}}\end{matrix} & (8) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}s\quad {i_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {q_{1} = {{q_{2}\bigwedge\alpha_{1}} = {{\alpha_{2}\bigwedge r_{1}} \geq_{\beta_{1}\bigwedge{\beta_{2}}}r_{2}}}}\end{matrix} & (9) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}{f_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {{q_{1} \geq_{{\alpha_{1}\bigwedge\alpha_{2}}}{q_{2}\bigwedge r_{1}}} = {{r_{2}\bigwedge\beta_{1}} = \beta_{2}}}}\end{matrix} & (10) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}f\quad {i_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {{q_{1} \leq_{\alpha_{1}\bigwedge{\alpha_{2}}}{q_{2}\bigwedge r_{1}}} = {{r_{2}\bigwedge\beta_{1}} = \beta_{2}}}}\end{matrix} & (11) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}{d_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {q_{1} \geq_{{\alpha_{1}\bigwedge\alpha_{2}}}{q_{2}\bigwedge r_{1}} \leq_{{\beta_{1}\bigwedge\beta_{2}}}r_{2}}}\end{matrix} & (12) \\\begin{matrix}{{{\quad \quad_{\alpha_{1}}}\left\lbrack {q_{1},r_{1}} \right\rbrack}_{\beta_{1}}d\quad {i_{\alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack}_{\beta_{2}}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {q_{1} \leq_{\alpha_{1}\bigwedge{\alpha_{2}}}{q_{2}\bigwedge r_{1}} \geq_{\beta_{1}\bigwedge{\beta_{2}}}r_{2}}}\end{matrix} & (13)\end{matrix}$

[0046] With the above definitions, exactly one Allen relation holdsbetween any pair of intervals.

[0047] The set of real numbers represented by an interval is referred toas its extension. Given the above definition of interval, any interval,such as [5, 4], (5, 4], [5, 4), or (5, 4), where the upper endpoint isless than the lower endpoint represents the empty set. And any openinterval, such as [5, 5), (5, 5], or (5, 5), where the upper endpointequals the lower endpoint also represents the empty set. To create asituation where the extension of each interval has a uniquerepresentation, let us represent all such empty sets of real numbers as{ }. Thus whenever we represent an interval _(α)[q, r]_(β) explicitly,it will have a nonempty extension and will satisfy the followingnormalization criterion: q≦

α

βr.

[0048] Spanning Intervals

[0049] When using event logic, we wish to compute and represent the setI of all intervals over which some event-logic expression Φ holds. Manyprimitive event types, including all of the primitive events types usedin the computer system implementation of the application described inthe “Example” section below, are liquid in the sense that if some eventholds of an interval then that event holds of every subinterval of thatinterval. With real-valued interval endpoints, this creates the need tocompute and represent an infinite set of intervals for a liquid event.Even limiting ourselves to integer-valued interval endpoints, a liquidevent will require the computation and representation of quadraticallymany intervals.

[0050] To address this problem, let us introduce the notion of aspanning interval. A spanning interval [i:j] represents the set of allsubintervals of [i, j], in other words {[q, r]|i≦q≦j

i≦r≦j}. Similarly (i:j], [i:j), and (i:j) represent {(q, r]|i≦q≦j

i≦r≦j}, {[q, r)|i≦q≦j

i≦r≦j}, and {(q, r)|i≦q≦j

i≦r≦j} respectively. What we desire is to use spanning intervals torepresent the set of all intervals over which the primitive event typeshold and to compute and represent the set of all intervals over whichcompound event types hold via structural induction over the compoundevent-logic expressions. A problem arises however. Given two liquidevent types Φ and Ψ, the compound event type Φ; Ψ is not liquid. If Φholds over [i:j) and Ψ holds over [j:k), then Φ; Ψ might not hold overevery subinterval of [i, k). It holds over only those subintervals thatinclude j. Such event types are referred to as semi liquid. Sincespanning intervals are not sufficient to efficiently representsemi-liquid events, let us extend the notion of a spanning interval. Aspanning interval [[i, j], [k, l]] represents the set of intervals {[q,r]|i≦q≦j

k≦r≦l}. Similarly the spanning intervals ([i, j], [k, l]], [[i, j], [k,l]), and ([i, j], [k, l]) represent the sets {(q, r]|i≦q≦j

k≦r≦l}, {[q, r)|i≦q≦j

k≦r≦l}, and {(q, r)|i≦q≦j

k≦r≦l} respectively. This extended notion of spanning interval subsumesthe original notion. The spanning intervals [i:j], (i:j], [i:j), and(i:j) can be represented as the spanning intervals [[i, j], [i, j]],([i, j], [i, j]], [[i, j], [i, j]), and ([i, j], [i, j]) respectively.For reasons that will become apparent below, it is necessary to alsoallow for spanning intervals where the ranges of endpoint values areopen. In other words, we will need to consider spanning intervals like[(i, j], [k, l]] to represent sets like {[q, r]|i≦q≦j

k≦r≦l}. All told, there are six endpoints that can independently beeither open or closed: q, r, i, j, k, and l, yielding sixty four kindsof spanning intervals. These can all be unified into a singlerepresentation, _(α)[_(γ)[i, j]_(δ), _(ε[k, l]) _(ζ)]_(β), where α, β,γ, δε, and ζ are true or false if the endpoints q, r, i, j, k, and l areclosed or open respectively. More precisely, the spanning interval_(α)[_(γ)[i, j]_(δ), _(ε)[k, l]_(ζ]) _(β) represents the set

{_(α)[q, r]_(β)|i≦_(γ)q≦_(δ)j

k≦_(ε)r≦_(ζ)l}  (14)

[0051] of intervals. The set of intervals represented by a spanninginterval is referred to as its extension. Moreover, a set of spanningintervals will represent the union of the extensions of its members andthe empty set of spanning intervals will represent the empty set ofintervals. The set of intervals represented by a set of spanningintervals is further referred to as its extension. A key result of thisdisclosure is that if the set of all intervals over which some set ofprimitive event types hold can be represented as finite sets of spanningintervals then the set of all intervals over which all event types thatare expressible as compound event-logic expressions over thoseprimitives hold can also be represented as finite sets of spanningintervals.

[0052] Normalizing Spanning Intervals

[0053] While we require that all intervals have finite endpoints, weallow spanning intervals to have infinite endpoints, for instance, [[−∞,j], [k, l]]. Such spanning intervals with infinite endpoints representsets of intervals with finite endpoints but where the range of possibleendpoints is unconstrained from above or below.

[0054] Just as we desire that the extension of every interval have aunique representation, we also desire that the extension of everyspanning interval have a unique representation. There are a number ofsituations where two different spanning intervals will have the sameextension. First, all spanning intervals _(α)[_(γ)[i, j]_(δ), _(ε[k, l])_(ζ)]_(β), where i=∞, j=−∞, k=∞, or l=−∞ represent the empty set ofintervals. Because there are no intervals with an endpoint that is lessthan or equal to minus infinity or greater than or equal to infinity.Second, if i=−∞, j=∞, k=−∞, or l=−∞, the value of γ, δ, ε, or ζ does notaffect the denotation respectively. Because there are no intervals withinfinite endpoints. Third, if j>l, j can be decreased as far as lwithout changing the denotation. Because all intervals where the upperendpoint is less than the lower endpoint equivalently denote the emptyinterval. Similarly, if k<i, k can be increased as far as i withoutchanging the denotation. Fourth, all spanning intervals where i>j or k>lrepresent the empty set of intervals. Because the range of possibleendpoints would be empty. Fifth, all spanning intervals where i=j andeither γ or δ is false (indicating an open range for the lower endpoint)represent the empty set of intervals. Because the range of possibleendpoints would be empty. Similarly, all spanning intervals where k=land either ε or ζ is false (indicating an open range for the upperendpoint) also represent the empty set of intervals. Sixth, all spanningintervals where i=l and either α or β is false (indicating an openinterval) also represent the empty set of intervals. Because theendpoints of an open interval must be different. Seventh, if j=l and ζis false, the value of a does not affect the denotation. Because if j=land ζ is false, the upper endpoint must be less than l and the lowerendpoint must be less than or equal to j which equals l, so the lowerendpoint must be less than j. Similarly, if k=i and γ is false, thevalue of ε does not affect the denotation. Eighth, if j=l and either αor β is false, the value of δ does not affect the denotation. Becausethe lower endpoint of an open interval must be less than its upperendpoint. Similarly, if k=i and either α or β is false, the value of εdoes not affect the denotation.

[0055] To create a situation where the extension of every spanninginterval has a unique representation, let us represent all empty sets ofintervals as { }. And when the values of i, j, k, l, α, β, γ, δ,ε, or ζcan be changed without changing the denotation, we will select thetightest such values. In other words, false values for the Booleanparameters, maximal values for the lower bounds, and minimal values forthe upper bounds. Thus whenever we represent a spanning interval_(α)[_(γ)[i, j]_(δ), _(ε)[k, l]_(ζ)]_(β) explicitly, it will have anonempty extension and will satisfy the following normalizationcriterion:

[0056] (1) i≠∞

j≠−∞

k≠∞

l≠−∞

[0057] (2) (i=−∞→

γ)

(j=∞→

δ)

(k=−∞→

ε)

(l=∞→

ζ)

[0058] (3) j≦l

k≧i

[0059] (4) i≦j

k≦l

[0060] (5) (i≠j

γ

δ)

(k≠l

ε

ζ)

[0061] (6) (i≠l

α

β)

[0062] (7) [(j=l

ζ)→

δ]

[(k=i

γ)→

ε]

[0063] (8) {[j=l

(

α

β)]→

δ}

{[k=i

(

α

β)]→

ε}

[0064] Criteria (1) through (8) correspond to points one through eightabove.

[0065] A spanning interval _(α)[_(γ)[i, j]_(δ), _(ε)[k, l]_(ζ)]_(β) isnormalized if i, j, k, l, α, β, γ, δ, ε, and ζ cannot be changed withoutchanging its denotation. Given a (potentially non-normalized) spanninginterval i, its normalization <i> is the smallest set of normalizedspanning intervals that represents the extension of i. One can compute<i> as follows:${\langle_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l} \right\rbrack_{\zeta}} \right\rbrack_{\beta}\rangle}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \left\{ \begin{matrix}{\quad \left\{ {}_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta^{\prime}}{,_{\in^{\prime}}}_{\quad}\left\lbrack {k^{\prime},l} \right\rbrack_{\zeta^{\prime}}} \right\rbrack_{\beta} \right\} \quad} \\\begin{matrix}{{{where}\quad j^{\prime}} = \quad {\min \left( {j,l} \right)}} \\{\quad {k^{\prime} = \quad {\max \left( {k,i} \right)}}} \\{\quad {\gamma^{\prime} = \quad {{\gamma\bigwedge i} \neq {- \infty}}}\quad} \\{\quad {\delta^{\prime} = \quad {{{\delta\bigwedge\min}\quad \left( {j,l} \right)} \neq {\infty\bigwedge\left( {j < {l\bigvee{\zeta\bigwedge\alpha\bigwedge\beta}}} \right)}}}\quad} \\{\quad {\in^{\prime}{= \quad {\in {{\bigwedge{\max \left( {k,i} \right)}} \neq {{- \infty}\bigwedge\left( {k > {i\bigvee{\gamma\bigwedge\beta\bigwedge\alpha}}} \right)}}}}}} \\{\quad {\zeta^{\prime} = \quad {{\zeta\bigwedge l} \neq \infty}}} \\{{{{when}{\quad \quad}i}\quad \leq \quad {j^{\prime}\bigwedge k^{\prime}} \leq {l\bigwedge}}\quad} \\{\quad {\left\lbrack {i = {j^{\prime}->\left( {\gamma^{\prime}\bigwedge\delta^{\prime}} \right)}} \right\rbrack\bigwedge\left\lbrack {k^{\prime} = {l->\left( {\in^{\prime}{\bigwedge\zeta^{\prime}}} \right)}} \right\rbrack\bigwedge}} \\{\quad {\left\lbrack {i = {l->\left( {\alpha\bigwedge\beta} \right)}} \right\rbrack\bigwedge}} \\{\quad {i \neq {\infty\bigwedge j^{\prime}} \neq {{- \infty}\bigwedge k^{\prime}} \neq {\infty\bigwedge l} \neq {- \infty}}}\end{matrix} \\{{\left\{ \quad \right\} \quad {otherwise}}{\quad \quad}}\end{matrix} \right.$

[0066] An important property of spanning intervals is that for anyspanning interval i, <i> contains at most one normalized spanninginterval.

[0067] Computing the Intersection of Two Normalized Spanning Intervals

[0068] Given two normalized spanning intervals i₁ and i₂, theirintersection i₁∩i₂ is a set of normalized spanning intervals whoseextension is the intersection of the extensions of i₁ and i₂. One cancompute i₁∩i₂ as follows:

[0069]_(α) ₁ [_(γ) ₁ [i₁, j₁]_(δ) ₁ , _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁∩_(α) ₂ [_(γ) ₂ [i₂, j₂]_(δ) ₂ , _(ε) ₂ [k₂, l₂]_(ζ) ₂ ]_(β) ₂ Δ

[0070] <_(α) ₁ [_(γ)[max (i₁, i₂), min (j₁, j₂)]_(δ), _(ε)[max (k₁, k₂),min (l₁, l₂)]_(ζ)]_(β) ₁ > $\begin{matrix}{{{where}\quad \gamma} = \quad \left\{ \begin{matrix}{\quad \gamma_{1}} & {\quad {i_{1} > i_{2}}} \\{\quad {\gamma_{1}\bigwedge\gamma_{2}}} & {\quad {i_{1} = i_{2}}} \\{\quad \gamma_{2}} & {\quad {i_{1} < i_{2}}}\end{matrix} \right.} \\{\delta = \quad \left\{ \begin{matrix}{\quad \delta_{1}} & {\quad {j_{1} < j_{2}}} \\{\quad {\delta_{1}\bigwedge\delta_{2}}} & {\quad {j_{1} = j_{2}}} \\{\quad \delta_{2}} & {\quad {j_{1} > j_{2}}}\end{matrix} \right.} \\{\in {= \quad \left\{ \begin{matrix}{\quad  \in_{1}} & {k_{1} > k_{2}} \\{\quad {\in_{1}{\bigwedge \in_{2}}}} & {k_{1} = k_{2}} \\{\quad  \in_{2}} & {k_{1} < k_{2}}\end{matrix} \right.}} \\{\zeta = \quad \left\{ \begin{matrix}{\quad \zeta_{1}} & {\quad {l_{1} < l_{2}}} \\{\quad {\zeta_{1}\bigwedge\zeta_{2}}} & {\quad {l_{1} = l_{2}}} \\{\quad \zeta_{2}} & {\quad {l_{1} > l_{2}}}\end{matrix} \right.}\end{matrix}\quad$

[0071] when α₁=α₂

β₁=β₂

[0072] { } otherwise An important property of normalized spanningintervals is that for any two normalized spanning intervals i₁ and i₂,i₁∩i₂ contains at most one normalized spanning interval.

[0073] The intuition behind the above definition is as follows. All ofthe intervals in the extension of a spanning interval are of the sametype, namely, [q, r], (q, r], [q, r), or (q, r). The intersection of twospanning intervals has a nonempty extension only if the two spanningintervals contain the same type of intervals in their extension. If theydo, and the sets contain intervals whose lower endpoint is bound frombelow by i₁ and i₂ respectively, then the intersection will containintervals whose lower endpoint is bound from below by both i₁ and i₂.The resulting bound is open or closed depending on which of the inputbounds is tighter. Similarly for the upper bound on the lower endpointand the lower and upper bounds on the upper endpoint.

[0074] Computing the Complement of a Normalized Spanning Interval

[0075] Given a normalized spanning interval i, its complement

i is a set of normalized spanning intervals whose extension is thecomplement of the extension of i. One can compute

i as follows:$_{\alpha}{\left\lbrack {}_{\gamma}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l} \right\rbrack_{\zeta}} \right\rbrack_{\beta}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \begin{pmatrix}{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},k} \right\rbrack_{- \in}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{\zeta}\left\lbrack {l,\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},i} \right\rbrack_{\gamma},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{\delta}{\left\lbrack {j,\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}}\end{pmatrix}}$

[0076] An important property of normalized spanning intervals is thatfor any normalized spanning interval i,

i contains at most seven normalized spanning intervals.

[0077] The intuition behind the above definition is as follows. Firstnote that the negation of q≦_(α)r is q≧

αr. Next note that the extension of i contains intervals whose endpointsq and r satisfy q≧_(γ)i

q≦_(δ)j

r≧_(ε)k

r≦_(ζ)l. Thus the extension of

i contains intervals whose endpoints satisfy the negation of this,namely, q≦

γi

q≧

δj

r≦

εk

r≦

ζl. Such a disjunction requires four spanning intervals, the first fourin the above definition. Additionally, if the extension of i containsintervals of the form [q, r], the extension of

i will contain all intervals not of the form [q, r], namely, (q, r], [q,r), and (q, r). Similarly for the cases where the extension of icontains intervals of the form (q, r], [q, r), or (q, r). This accountsfor the last three spanning intervals in the above definition.

[0078] We now see why it is necessary to allow spanning intervals tohave open ranges of endpoint values. The complement of a spanninginterval, such as [[i, j],[k, l]], with closed endpoint ranges, includesspanning intervals, such as [[−∞, i), [−∞, ∞]], with open endpointranges.

[0079] Computing the Span of Two Normalized Spanning Intervals

[0080] The span of two intervals i₁ and i₂, denoted SPAN (i₁, i₂), isthe smallest interval whose extension contains the extensions of both i₁and i₂. For example, the span of (1, 4) and [2, 6] is (1, 6]. And thespan of [3, 7) and (3, 7] is [3, 7]. More generally, the lower endpointof SPAN (i₁, i₂) is the minimum of the lower endpoints of i₁ and i₂. Andthe lower endpoint of SPAN (i₁, i₂) is open or closed depending onwhether the smaller of the lower endpoints of i₁ and i₂ is open orclosed. Analogously, the upper endpoint of SPAN (i₁, i₂) is the maximumof the upper endpoints of i₁ and i₂. And the upper endpoint of SPAN (i₁,i₂) is open or closed depending on whether the larger of the upperendpoints of i₁ and i₂ is open or closed. More precisely, SPAN (i₁, i₂)can be computed as follows:${{SPAN}\left( {}_{\alpha_{1}}{\left\lbrack {q_{1},r_{1}} \right\rbrack_{\beta_{1}},_{\quad \alpha_{2}}\left\lbrack {q_{2},r_{2}} \right\rbrack_{\beta_{2}}} \right)}{{\underset{\underset{\_}{\_}}{\Delta}}_{{({{a_{1}\bigwedge q_{1}} \leq q_{2}})}\bigvee{({{a_{2}\bigwedge q_{1}} \geq q_{2}})}}\left\lbrack {{\min \left( {q_{1},q_{2}} \right)},{\max \left( {r_{1},r_{2}} \right)}} \right\rbrack}_{{({{\beta_{1}\bigwedge r_{1}} \geq r_{2}})}\bigvee{({{\beta_{2}\bigwedge r_{1}} \leq r_{2}})}}$

[0081] The notion of span will be used below.

[0082] Let us extend the notion of span to two sets of intervals by thefollowing definition:${{{SPAN}\left( {I_{1},I_{2}} \right)}\underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i_{1} \in I_{1}}{\bigcup\limits_{i_{2} \in I_{2}}{{SPAN}\left( {i_{1},i_{2}} \right)}}$

[0083] We will want to compute the span of two sets of intervals I₁ andI₂, when both I₁ and I₂ are represented as spanning intervals. And wewill also want the resulting span to be represented as a small set ofspanning intervals.

[0084] Given two normalized spanning intervals i₁ and i₂, their spanSPAN (i₁, i₂) is a set of normalized spanning intervals whose extensionis the span of the extensions of i₁ and i₂. One can compute SPAN (i₁,i₂) as follows:${SPAN}\quad \left( {}_{\alpha_{1}}{\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1},\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j_{2}} \right\rbrack_{\delta_{2}},_{\in_{2}}\left\lbrack {k_{2},l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack}_{\beta_{2}} \right)\quad {\underset{\underset{\_}{\_}}{\Delta}\begin{pmatrix}{\quad {{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack_{\beta_{2}}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}\bigcup}} \\{\quad {\langle_{\alpha}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack_{\beta_{2}}\rangle}}\end{pmatrix}}$

[0085] where

[0086] j=min (j₁, j₂)

[0087] k=max (k₁, k₂)

[0088] δ=[(δ₁

j₁≦j₂)

(δ₂

j₁≧j₂)]

[0089] ε=[(ε₁

k₁≧k₂)

(ε₂

k₁≦k₂)]

[0090] An important property of normalized spanning intervals is thatfor any two normalized spanning intervals i₁ and i₂, SPAN (i₁, i₂)contains at most four normalized spanning intervals. In practice,however, fewer normalized spanning intervals are needed, often only one.

[0091] The intuition behind the above definition is as follows. Considerfirst the lower endpoint. Suppose that the lower endpoints q₁ and q₂ ofi₁ and i₂ are in _(γ) ₁ [i₁, j₁]_(δ) ₁ and _(γ) ₂ [i₂, j₂]_(δ) ₂respectively. That means that i₁≦_(γ) ₁ q₁≦_(δ) ₁ j₁ and i₂≦_(γ) ₂q₂≦_(δ) ₂ j₂. The lower endpoint of SPAN (i₁, i₂) will be q₁, whenq₁≦q₂, and q₂, when q₂≦q₁. Thus it will be q₁, for all i₁≦_(γ) ₁ q₁≦_(δ)min (j₁, j₂), and will be q₂, for all i₂≦_(γ) ₂ q₂≦_(δ) min (j₁, j₂),where δ=δ₁, when j₁≦j₂, and δ=δ₂, when j₁≧j₂. Thus there will be twopotential ranges for the lower endpoint of SPAN (i₁, i₂): _(γ) ₁ [i₁,min (j₁, j₂)]_(δ) and _(γ) ₂ [i₂, min (j₁, j₂)]_(δ). When the lowerendpoint of SPAN (i₁, i₂) is taken from the former, it will be open orclosed depending on whether the lower endpoint of i₁ is open or closed.When it is taken from the later, it will be open or closed depending onwhether the lower endpoint of i₂ is open or closed. Thus the lowerendpoint of SPAN (i₁, i₂) can be either _(α) ₁ [_(γ) ₁ [i₁, min (j₁,j₂)]₆₇ or _(α) ₁ [_(γ) ₂ [i₂, min (j₁, j₂ )]_(δ). Analogous reasoningcan be applied to the upper endpoints. If the upper endpoints of i₁ andi₂ are _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁ and _(ε) ₂ [k₂, l₂]_(ζ) ₂ ]_(β) ₂respectively, then there are two possibilities for the upper endpoint ofSPAN (i₁, i₂), namely, _(ε)[max (k₁, k₂), l₁]_(ζ) ₁ ]_(β) ₁ and _(ε)[max(k₁,k₂), l₂]_(ζ) ₂ ]_(β) ₂ where ε=ε₁, when k₁≧k₂, and ε=ε₂, when k₁≦k₂.

[0092] Computing the

of a Normalized Spanning Interval

[0093] Given an Allen relation r and a set I of intervals, let

(r, I) denote the set J of all intervals j such that irj for some iεI.Given an Allen relation r and a normalized spanning interval i, let

(r, i) denote a set of normalized spanning intervals whose extension is

(r, I), where I is the extension of i. One can compute

(r, i) as follows: $\begin{matrix}{\quad {{\quad }\left( {= i} \right)}\quad} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad \left\{ i \right\}} \\{{\quad {\left( {< {,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}}} \right)}}\quad} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{{\beta_{1}\bigwedge{{\alpha_{2}\bigwedge\varepsilon_{1}}}}}{\left\lbrack {k_{1},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {> {,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}}} \right)}\quad} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},j_{1}} \right\rbrack}_{{\alpha_{1}\bigwedge{{\beta_{2}\bigwedge\delta_{1}}}}} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {m,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\beta_{1}}\left\lbrack {}_{\quad \varepsilon_{1}}{\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1},T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {{m\quad i},_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,\gamma_{1}}\left\lbrack {i_{1},j_{1}} \right\rbrack}_{\delta_{1}} \right\rbrack_{\alpha_{1}}\rangle}}} \\{\quad {\left( {o,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{\quad {\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}}{\left\lbrack {i_{1},l_{1}} \right\rbrack_{{\beta_{1}\bigwedge\alpha_{2}\bigwedge\zeta_{1}},{{\beta_{1}\bigwedge\beta_{2}\bigwedge\varepsilon_{1}}}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {{oi},_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},{\alpha_{1}\bigwedge\beta_{2}\bigwedge\gamma_{1}}}\left\lbrack {i_{1},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {s,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\quad \alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{{\beta_{1}\bigwedge\beta_{2}\bigwedge\varepsilon_{1}}}\left\lbrack {k_{1},\infty} \right\rbrack_{T}} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {{si},_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\quad \alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},{{}_{}^{}\left\lbrack {{- \infty},l_{1}} \right\rbrack_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}}^{}}} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {f,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}\rangle}}} \\{\quad {\left( {{fi},_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{\quad {\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}}{\left\lbrack {i_{1},\infty} \right\rbrack_{T},_{\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}}} \\{\quad {\left( {d,_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},},_{{\beta_{1}\bigwedge\beta_{2}\bigwedge\varepsilon_{1}}}\left\lbrack {k_{1},\infty} \right\rbrack_{T}} \right\rbrack_{\beta_{2}}\rangle}}} \\{\quad {\left( {{di},_{\alpha_{1}}\left\lbrack {}_{\quad \gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}} & {\quad \underset{\underset{\_}{\_}}{\Delta}} & {\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\quad \alpha_{2}}\left\lbrack {}_{\quad {\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}}{\left\lbrack {i_{1},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack_{\beta_{2}}\rangle}}}\end{matrix}$

[0094] An important property of normalized spanning intervals is thatfor any normalized spanning interval i,

(r, i) contains at most 1, 4, 4, 2, 2, 4, 4, 2, 2, 2, 2, 4, or 4normalized spanning intervals when r is =,<,>, m, mi, o, oi, s, si, f,fi, d, or di respectively. In practice, however, fewer normalizedspanning intervals are needed, often only one.

[0095] The intuition behind the above definition is as follows. Let ushandle each of the cases separately.

[0096] r=< For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁<i′₂. From (2) we get r₁≦

β ₁

α ₂ q₂. And from (14) we get k₁≦_(ε) ₁ r₁. Combining these we get k₁≦

β ₁

α ₂

ε ₁ q₂. In this case, both α₂ and β₂ are free indicating that eitherendpoint of i′₂ can be open or closed.

[0097] r=> For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁>i′₂. From (3) we get q₁≧

α ₁

β ₂ r₂. And from (14) we get q₁≦_(δ) ₁ j₁. Combining these we get r₂≦

α ₁

β ₂

δ ₁ j₁. In this case, both α₂ and β₂ are free indicating that eitherendpoint of i′₂ can be open or closed.

[0098] r=m For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ mi′₂. From (4) we get r₁=q₂and β₁≠α₂. And from(14) we get k₁≦_(ε) ₁ r₁≦_(ζ) ₁ l₁. Combining these we get k₁≦_(ε) ₁q₂≦_(ζ) ₁ l₁ and β₁≠α₂. In this case, only β₂ is free indicating thatthe upper endpoint of i′₂ can be open or closed.

[0099] r=mi For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ mi i′₂. From (5) we get q₁=r₂ and α₁≠β₂. Andfrom (14) we get i₁≦_(γ) ₁ q₁≦_(δ) ₁ j₁. Combining these we get i₁≦_(γ)₁ r₂≦_(δ) ₁ j₁ and α₁≠β₂. In this case, only α₂ is free indicating thatthe lower endpoint of i′₂ can be open or closed.

[0100] r=o For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ oi′₂. From (6) we get q₁≦_(α) ₁

α ₂ q₂≦_(β) ₁

α ₂ r₁≦

β ₁

β ₂ r₂. And from (14) we get i₁≦_(γ) ₁ q₁ and k₁≦_(ε) ₁ r₁≦_(ζ) ₁ l₁.Combining these we get i₁≦_(α) ₁

α ₂

γ ₁ q₂≦_(β) ₁

α ₂

ζ ₁ l₁ and k₁≦

β ₁

β ₂

ε ₁ r₂. In this case, both α₂ and β₂ are free indicating that eitherendpoint of i′₂ can be open or closed.

[0101] r=oi For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ oi i′₂. From (7) we get q₂≦

α ₁

α ₂ q₁≦_(α) ₁ _(β) ₂ r₂≦_(β) ₁

β ₂ r₁. And from (14) we get r₁≦_(ζ) ₁ l₁ and i₁≦_(γ) ₁ q₁≦_(δ) ₁ j₁.Combining these we get i₁≦_(α) ₁

β ₂

γ ₁ r₂≦_(β) ₁

β ₂

ζ ₁ l₁ and q₂≦

α ₁

α ₂

δ ₁ j₁. In this case, both α₂ and β₂ are free indicating that eitherendpoint of i′₂ can be open or closed.

[0102] r=s For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ s i′₂. From (8) we get q₁=q₂, α₁=α₂, and r₁≦

β ₁

β ₂ r₂. And from (14) we get i₁≦_(γ) ₁ q₁≦_(δ) ₁ j₁ and k₁≦_(ε) ₁ r₁.Combining these we get α₁=α₂, i₁≦_(γ) ₁ q₂≦_(δ) ₁ j₁, and k₁≦

β ₁

β ₂

ε ₁ r₂. In this case, only β₂ is free indicating that the upper endpointof i′₂ can be open or closed.

[0103] r=si For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ si i′₂. From (9) we get q₁=q₂, α₁=α₂, andr₁≧_(β) ₁

β ₂ r₂. And from (14) we get i₁≦_(γ) ₁ q₁≦_(δ) ₁ j₁ and r₁≦_(ζ) ₁ l₁.Combining these we get α₁=α₂, i₁≦_(γ) ₁ q₂≦_(δ) ₁ j₁, and r₂≦_(β) ₁

β ₂

ζ ₁ l₁. In this case, only β₂ is free indicating that the upper endpointof i′₂ can be open or closed.

[0104] r=f For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ f i′₂. From (10) we get q₁≧

α ₁

α ₂ q₂, r₁=r₂, and β₁=β₂. And from (14) we get k₁≦_(ε) ₁ r₁≦_(ζ) ₁ l₁and q₁≦_(δ) ₁ j₁. Combining these we get β₁=β₂, k₁≦_(ε) ₁ r₂≦_(ζ) ₁ l₁,and q₂≦

α ₁

α ₂

δ ₁ j₁. In this case, only α₂ is free indicating that the lower endpointof i′₂ can be open or closed.

[0105] r=fi For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ fi i′₂. From (11) we get q₁≧_(α) ₁

α ₂ q₂, r₁=r₂, and β₁=β₂. And from (14) we get k₁≦_(ε) ₁ r₁≦_(ζ) ₁ l₁and i₁≦_(γ) ₁ q₁. Combining these we get β₁=β₂, k₁≦_(ε) ₁ r₂≦_(ζ) ₁ l₁,and i₁≦_(α) ₁

α ₂

γ ₁ q₂. In this case, only α₂ is free indicating that the lower endpointof i′₂ can be open or closed.

[0106] r=d For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ di′₂. From (12) we get q₁≧

α ₁

α ₂ q₂ and r₁≦_(β) ₂

β ₂ r₂. And from (14) we get q₁≦_(δ) ₁ j₁ and k₁≦_(ε) ₁ r₁. Combiningthese we get q₂≦

α ₁ _(α) ₂

δ ₁ j₁ and k₁≦

β ₁

β ₂

ε ₁ r₂. In this case, both α₂ and β₂ are free indicating that eitherendpoint of i′₂ can be open or closed.

[0107] r=di For any intervals i′₁ and i′₂ in the extensions of i₁ and i₂respectively we want i′₁ di i′₂. From (13) we get q₁≦_(α) ₁

α ₂ q₂, and r₁≧_(β) ₁

β ₂ r₂. And from (14) we get i₁≦_(γ) ₁ q₁ and r₁≦_(ζ) ₁ l₁. Combiningthese we get i₁≦_(α) ₁

α ₂

γ ₁ q₂ and r₂≦_(β) ₁

β ₂

ζ ₁ l₁. In this case, both α₂ and β₂ are free indicating that eitherendpoint of i′₂ can be open or closed.

[0108] Computing the

of Two Normalized Spanning Intervals

[0109] Given an Allen relation r and two sets I and J of intervals, let

(I, r, J) denote the set K of all intervals k such that k=SPAN (i, j)for some iεI and jεJ, where irj. Given an Allen relation r and twonormalized spanning intervals i and j, let

(i, r, j) denote a set of normalized spanning intervals whose extensionis

(I, r, J), where I and J are the extensions of i and j respectively. Onecan compute

(i, r, j) as follows:${\left. {\left( {i,r,j} \right.} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i^{\prime}\varepsilon {{({r^{- 1},j})}}}{\bigcup\limits_{{i^{\prime}\varepsilon \quad i^{\prime}}\bigcap i}{\bigcup\limits_{j^{\prime}\varepsilon {{({r,i})}}}{\bigcup\limits_{{j^{\prime}\varepsilon \quad j^{\prime}}\bigcap j}{{SPAN}\left( {i^{''},j^{''}} \right)}}}}$

[0110] It is easy to see that |

(., r,.)|≦4|

(r, .)|². Thus an important property of normalized spanning intervals isthat for any two normalized spanning intervals i and j,

(i, r, j) contains at most 4, 64, 64, 16, 16, 64, 64, 16, 16, 16, 16,64, or 64 normalized spanning intervals, when r is =, <, >, m, mi, o,oi, s, si, f, fi, d, or di respectively. While simple combinatorialenumeration yields the above weak bounds on the number of normalizedspanning intervals needed to represent

(i, r, j), in practice, far fewer normalized spanning intervals areneeded, in most cases only one.

[0111] The intuition behind the above definition is as follows. Let Iand J be the extensions of i and j respectively. The extension of theset of all i′ is the set of all intervals i such that irj for some j inJ. And the extension of the set of all i″ is the set of all intervals iin I such that irj for some j in J. Similarly, the extension of the setof all j′ is the set of all intervals j such that irj for some i in I.And the extension of the set of all j″ is the set of all intervals j inJ such that irj for some i in I. Thus the extension of the set of allSPAN (i″ j″) is the set of all intervals k such that k=SPAN (i, j) wherei is in I, j is in J, and irj.

[0112] An Efficient Inference Procedure for Event Logic

[0113] Given the above procedures for computing <i>, i₁∩i₂,

i, SPAN (i₁, i₂),

(r, i), and

(i, r, j), one can now define a procedure for computing ε(M, Φ). Thisprocedure takes a model M along with an event-logic expression Φ andcomputes a set of normalized spanning intervals that represents the setI of intervals i for which Φ@i is true. The model M is a set of atomicevent-occurrence formulae of the form p(c₁, . . . , c_(n))@i, wherep(c₁, . . . , c_(n)) is a ground primitive event-logic expression and iis a normalized spanning interval. A model entry p(c₁, . . . , c_(n))@iindicates that the primitive event p(c₁, . . . , c_(n)) occurred duringall intervals in the extension of i. C(M) denotes the set of allconstants in all ground primitive event-logic expressions in M.$\begin{matrix}{ɛ\left( {M,{p\left( {c_{1},\ldots \quad,c_{n}} \right)}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & \left\{ i \middle| {{{p\left( {c_{1},\ldots \quad,c_{n}} \right)}@i}\quad \varepsilon \quad M} \right\} \\{ɛ\left( {M,{\Phi\bigvee\Psi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {{ɛ\left( {M,\Phi} \right)}\bigcup{ɛ\left( {M,\Psi} \right)}} \\{ɛ\left( {M,{\forall{x\quad \Phi}}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {{\bigcup\limits_{i_{1}\varepsilon \quad {ɛ{({M,{\Phi {\lbrack{x:=c_{1}}\rbrack}}})}}}\ldots}\bigcup\limits_{i_{n}\varepsilon \quad {ɛ{({M,{\Phi {\lbrack{x:=c_{n}}\rbrack}}})}}}{i_{1}\bigcap\ldots\bigcap i_{n}}} \\\quad & \quad & {{{where}\quad {C(M)}} = \left\{ {c_{1},\ldots \quad,c_{n}} \right\}} \\{ɛ\left( {M,{\exists{x\quad \Phi}}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {\bigcup\limits_{c\quad \varepsilon \quad {C{(M)}}}{ɛ\left( {M,{\Phi \left\lbrack {x:=c} \right\rbrack}} \right)}} \\{ɛ\left( {M,{\Phi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {{\bigcup\limits_{i_{1}^{\prime}\varepsilon {i_{1}}}\ldots}\bigcup\limits_{i_{n}^{\prime}\varepsilon {\quad i_{n}}}{i_{1}^{\prime}\bigcap\ldots\bigcap i_{n}^{\prime}}} \\\quad & \quad & {{{where}\quad {ɛ\left( {M,\Phi} \right)}} = \left\{ {i_{1},\ldots \quad,i_{n}} \right\}} \\{ɛ\left( {M,{\Phi \bigwedge_{R}\Psi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {\bigcup\limits_{i\quad \varepsilon \quad {ɛ{({M,\Phi})}}}{\bigcup\limits_{j\quad \varepsilon \quad {ɛ{({M,\Psi})}}}{\bigcup\limits_{r\quad \varepsilon \quad R}{\left( {i,r,j} \right)}}}} \\{ɛ\left( {M,{\Diamond_{R}\Phi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {\bigcup\limits_{i\quad \varepsilon \quad {ɛ{({M,\Phi})}}}{\bigcup\limits_{r\quad \varepsilon \quad R}{\left( {r,i} \right)}}}\end{matrix}$

[0114] The procedure performs structural induction on Φ as set forth inmore detail in FIG. 2. It computes a set of normalized spanningintervals to the represent the occurrence of each atomic event-logicexpression in Φ and recursively combines the sets so computed for eachchild subexpression to yield the sets for each parent subexpression. Animportant property of this inference procedure is that for any finitemodel M, ε(M, Φ), the set I of intervals i for which Φ@i is true, can berepresented by a finite set of normalized spanning intervals. Nominally,the number of normalized spanning intervals in ε(M, Φ) can beexponential in the subexpression depth of Φ because each step in thestructural induction can introduce a constant factor growth in the sizeof the set. However, in practice, such exponential growth does notoccur.

EXAMPLE

[0115] The methods of the present invention have been implemented as acomputer system for recognizing events in video sequences. However, therecognition of events in video sequences is given by way of example onlyand not to limit the scope or spirit of the present invention. Thoseskilled in the art will realize that there are many other applicationsfor the methods of the present invention.

[0116] The computer system takes short (typically 30 to 120 frame) videosequences as input. These video sequences depict a person performingvarious actions with colored blocks, such as pick up, put down, stack,unstack, move, assemble, and disassemble. A video sequence can depict nodefined action, one defined action, or multiple defined actions.Multiple defined actions may be sequential and/or simultaneous and maybe the same action or different actions. The computer system labels eachvideo sequence with the actions that are being performed as well as theparticular interval in the video sequence during which those actionswere performed.

[0117] This computer system uses the methods of the present invention toperform event classification. FIG. 3 shows the primitive event typesused by this computer system. The intervals in the input video sequencesduring which these primitive event types hold are computed usingtechniques in the prior art. These intervals, however, are representedas spanning intervals introduced by the present invention. Thisspecification of the primitive event types and the mechanism forcomputing the intervals corresponding to their occurrence in input videosequences constitutes an application of steps 102 and 106 as shown inFIG. 1. FIG. 4 shows the compound event types used by this computersystem. These compound event types are specified as event-logicexpressions over the primitive event types from FIG. 3. Thisspecification of the compound event types constitutes an application ofstep 104 as shown in FIG. 1.

[0118] PICKUP (x, y, z) denotes an event type where x picks y up off ofz. It is specified as a sequence of three intervals, where x is notattached to and does not support y in the first interval but is attachedto and does support y in the third interval. And z supports y in thefirst interval but does not support y in the third interval.Additionally, several conditions must hold in both the first and thirdintervals: x must be unsupported, y must not support either z or x, xand z must not support each other, and y must not be attached to z.During the second interval, intermediate between the first and thirdintervals, either x is attached to y or y is attached to z.Additionally, several conditions must hold throughout the entire event:x, y, and z must be distinct and y must be supported. PUTDOWN (x, y, z)denotes an event type where x puts y down on z. It is specified in afashion that is similar to PICKUP (x, y, z) but where the threesubevents occur in reverse order. STACK (w, x, y, z) denotes an eventtype where w puts x down on y which is resting on z. It is specified asPUTDOWN (w, x, y), where z supports but is not attached to y and z isdistinct from w, x, and y. UNSTACK (w, x, y, z) denotes an event typewhere w picks x up off of y which is resting on z. It is specified asPICKUP (w, x, y), where z supports but is not attached to y and z isdistinct from w, x, and y. MOVE (w, x, y, z) denotes an event type wherew picks x up off of y and puts it down on z which is distinct from y.ASSEMBLE (w, x, y, z) denotes an event type where w first puts y down onz then sometime later stacks x on top of y. Finally, DISASSEMBLE (w, x,y, z) denotes an event type where w first unstacks x from on top of y(which is resting on z) and then sometime later picks y up off of z.FIGS. 5A and 5B show sample movies depicting occurrences of the eventtypes PICKUP (x, y, z) and PUTDOWN (x, y, z), respectively. FIGS. 7A-7Eshow sample movies depicting occurrences of the event types STACK (w, x,y, z), UNSTACK (w, x, y, z), MOVE (w, x, y, z), ASSEMBLE (w, x, y, z),and DISASSEMBLE (w, x, y, z), respectively.

[0119] Nominally, all atomic event-logic expressions are primitive eventtypes. However, we allow giving a name to a compound event-logicexpression and using this name in another event-logic expression asshort hand for the named expression with appropriate parametersubstitution. This is simply a macro-expansion process and, as such, norecursion is allowed. This feature is used in FIG. 4 to define UNSTACK,MOVE, and DISASSEMBLE in terms of PICKUP, STACK, MOVE, and ASSEMBLE interms of STACK, which is itself defined in terms of PUTDOWN, andDISASSEMBLE in terms of UNSTACK, which is itself defined in terms ofPICKUP.

[0120] The methods of the present invention have been applied to computethe occurrences of the compound event types from FIG. 4 from theoccurrences of the primitive event types from FIG. 3 as recovered from anumber of input video sequences. This constitutes an implementation ofstep 108 as shown in FIG. 1. FIGS. 6, 8, 10, and 12 show both theprimitive event occurrences that have been recovered from the inputvideo sequences in FIGS. 5, 7, 9, and 11 using methods in the prior artas well as the compound event occurrences that have been recognized outof the primitive event occurrences using the methods of the presentinvention. FIGS. 5A, 5B, and 7A-7E show sample movies that depict theseven compound event types pick up, put down, stack, unstack, move,assemble, and disassemble respectively. The results of applyingsegmentation, tracking, and model reconstruction methods of the priorart on these video sequences are shown overlayed on the video sequences.FIGS. 6A, 6B, and 8A-8E show the results of applying the eventclassification methods of the present invention on these movies. Thesefigures show that the computer system implementation of the methods ofthe present invention correctly recognized the intended event class foreach movie.

[0121] In FIG. 5A, frames 0 through 1 correspond to the first subeventof a pick up event, frames 2 through 13 correspond to the secondsubevent, and frames 14 through 22 correspond to the third subevent. InFIG. 5B, frames 0 through 13 correspond to the first subevent of a putdown event, frames 14 through 22 correspond to the second subevent, andframes 23 through 32 correspond to the third subevent. The computersystem correctly recognized these as instances of pick up and put downrespectively. In FIG. 7A, frames 0 through 11, 12 through 23, and 24through 30 correspond to the three subevents of a put down event. Thecomputer system correctly recognized this as a put down event and alsoas a stack event. In FIG. 7B, frames 0 through 10, 11 through 24, and 25through 33 correspond to the three subevents of a pick up event. Thecomputer system correctly recognized this as a pick up event and also asan unstack event. In FIG. 7C, frames 0 through 8, 9 through 16, and 17through 45 correspond to the three subevents of a pick up event andframes 17 through 33, 34 through 45, and 46 through 52 correspond to thethree subevents of aput down event. The computer system correctlyrecognized the combination of these two events as a move event. In FIG.7D, frames 18 through 32, 33 through 40, and 41 through 46 correspond tothe three subevents of a put down event and frames 57 through 67 and 68through 87 correspond to the first and third subevents of a second putdown event, with the second subevent being empty. The latter put downevent was also correctly recognized as a stack event and the combinationof these two events was correctly recognized as an assemble event. InFIG. 7E, frames 0 through 18, 19 through 22, and 23 through 50correspond to the three subevents of a pick up event and frames 23through 56, 57 through 62, and 63 through 87 correspond to the threesubevents of a second pick up event. The former pick up event was alsocorrectly recognized as an unstack event and the combination of thesetwo events was correctly recognized as a disassemble event. Theseexamples show that the computer system correctly recognized each of theseven event types with no false positives.

[0122] As discussed in the introduction, using force dynamics and eventlogic to recognize events offers several advantages over the prior artof using motion profile and hidden Markov models. These advantages are:

[0123] (1) robustness against variance in motion profile;

[0124] (2) robustness against presence of extraneous objects in thefield of view;

[0125] (3) ability to perform temporal and spatial segmentation ofevents; and

[0126] (4) ability to detect non-occurrence of events.

[0127] FIGS. 9A-9D and 10A-10D illustrate the first three of theseadvantages while FIGS. 11A, 11B, 12A, and 12B illustrate the third. FIG.9A shows a pick up event from the left in contrast to FIG. 5A which isfrom the right. Even though these have different motion profiles, FIG.10A shows that the computer system correctly recognized that theseexhibit the same sequence of changes in force-dynamic relations andconstitute the same event type, namely pick up. FIG. 9B shows a pick upevent with two extraneous blocks in the field of view. FIG. 10B showsthat the computer system correctly recognized that these extraneousblocks do not participate in any events and, despite their presence, thetruth conditions for a pick up event still hold between the otherobjects. FIG. 9C shows a pick up event, followed by a put down event,followed by another pick up event, followed by another put down event.FIG. 10C shows that the computer system correctly recognizes thissequence of four event occurrences. FIG. 9D shows two simultaneous pickup events. FIG. 10D shows that the computer system correctly recognizedthese two simultaneous event occurrences. Finally, FIGS. 11A and 11Bshow two non-events. FIGS. 12A and 12B show that the computer system isnot fooled into thinking that these constitute pick up or put downevents, even though portions of these events have similar motion profileto pick up and put down events. Therefore, the computer system correctlyrecognizes that these movies do not match any known event types.

[0128] The methods of the present invention are incorporated in acomprehensive implemented system for recovering event occurrences fromvideo input. It differs from prior approaches to the same problem in twofundamental ways. It uses state changes in the force-dynamic relationsbetween objects, instead of motion profile, as the key descriptiveelement in defining event types. And it uses event logic, instead ofhidden Markov models, to perform event classification. One key result ofthe methods of the present invention is the formulation of spanningintervals, a novel efficient representation of the infinite sets ofintervals that arise when processing liquid and semi-liquid events. Asecond key result is the formulation of an efficient procedure, based onspanning intervals, for inferring all occurrences of compound eventtypes from occurrences of primitive event types. The techniques offorce-dynamic model reconstruction, spanning intervals, and event-logicinference have been used to successfully recognize seven event typesfrom real video: pick up, put down, stack, unstack, move, assemble, anddisassemble. Using force-dynamics and event logic to perform eventrecognition offers four key advantages over the prior art of usingmotion profile and hidden Markov models. First, it is insensitive tovariance in the motion profile of an event occurrence. Second, it isinsensitive to the presence of extraneous objects in the field of view.Third, it allows temporal segmentation of sequential and parallel eventoccurrences. And fourth, it robustly detects the non-occurrence ofevents as well as their occurrence.

[0129] As discussed above, the methods of the present invention areparticularly suited to be carried out by a computer software program,such computer software program preferably containing modulescorresponding to the individual steps of the methods. Such software can,of course, be embodied in a computer-readable medium, such as anintegrated circuit or a peripheral device.

[0130] While there has been shown and described what is considered to bepreferred embodiments of the invention, it will, of course, beunderstood that various modifications and changes in form or detailcould readily be made without departing from the spirit of theinvention. It is therefore intended that the invention be not limited tothe exact form described and illustrated, but should be constructed tocover all modifications that may fall within the scope of the appendedclaims.

What is claimed is:
 1. A method for computing all occurrences of acompound event from occurrences of primitive events where the compoundevent is a defined combination of the primitive events, the methodcomprising the steps of: (a) defining primitive event types; (b)defining combinations of the primitive event types as a compound eventtype; (c) inputting the primitive event occurrences, such occurrencesbeing specified as the set of temporal intervals over which a givenprimitive event type is true; and (d) computing the compound eventoccurrences, such occurrences being specified as the set of temporalintervals over which the compound event type is true, wherein the setsof temporal intervals in steps (c) and (d) are specified as smaller setsof spanning intervals, each spanning interval representing a set ofintervals.
 2. The method according to claim 1, wherein the spanningintervals take the form _(α)[_(γ)[i, j]_(δ), _(ε)[k, l]_(ζ)]_(β), whereα, β, γ, δ, ε, and ζ are Boolean values, i, j, k, and l are realnumbers, _(α)[_(γ)[i, j]_(δ), _(ε)[k, l]_(ζ) 9 _(β) represents the setof all intervals _(α)[p, q]_(β) where i≦_(γ)p≦_(δ)j and k≦_(ε)q≦_(ζ)l,_(α)[a p, q]_(β) represents the set of all points r, wherep≦_(α)r<_(β)q, and x≦_(θ) y means x≦y when θ is true and x<y when θ isfalse.
 3. The method according to claim 2, wherein the compound eventtype in step (b) is specified as an expression in temporal logic.
 4. Themethod according to claim 3, wherein the temporal logic expressions areconstructed using the logical connectives ∀, ∃,

,

_(R), ⋄_(R), and

, where R ranges over sets of relations between one-dimensionalintervals.
 5. The method according to claim 4, wherein the relations are=, <, >, m, mi, o, oi, s, si, f, fi, d, and di.
 6. The method accordingto claim 5, wherein the compound event occurrences are computed usingthe following set of equations:${ɛ\left( {M,{p\left( {c_{1},\ldots \quad,c_{n}} \right)}} \right)}\underset{\underset{\_}{\_}}{\Delta}\left\{ i \middle| {{{p\left( {c_{1},\ldots \quad,c_{n}} \right)}@i} \in M} \right\}$${{ɛ\left( {M,{\Phi\bigvee\Psi}} \right)}\underset{\underset{\_}{\_}}{\Delta}{ɛ\left( {M,\Phi} \right)}}\bigcup{ɛ\left( {M,\Psi} \right)}$${{ɛ\left( {M,{\forall{x\quad \Phi}}} \right)}\underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i_{1} \in {ɛ{({M,{\Phi {\lbrack{x:=c_{1}}\rbrack}}})}}}\ldots \bigcup\limits_{i_{1} \in {ɛ{({M,{\Phi {\lbrack{x:=c_{n}}\rbrack}}})}}}{i_{1}\bigcap\ldots\bigcap i_{n}}$where  C(M) = {c₁, …  , c_(n)}${{ɛ\left( {M,{\exists{x\quad \Phi}}} \right)}\underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{c \in {C{(M)}}}{ɛ\left( {M,{\Phi \left\lbrack {x:=c} \right\rbrack}} \right)}$${{ɛ\left( {M,{\quad \Phi}} \right)}\underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i_{1}^{\prime} \in _{1}}\ldots \bigcup\limits_{i_{n}^{\prime} \in _{n}}{i_{1}^{\prime}\bigcap\ldots\bigcap i_{n}^{\prime}}$where  ɛ(M, Φ) = {i₁, …  , i_(n)}${{ɛ\left( {M,{\Phi \bigwedge_{R}\Psi}} \right)}\underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i \in {ɛ{({M,\Phi})}}}{\bigcup\limits_{j \in {ɛ{({M,\Psi})}}}{\bigcup\limits_{r \in R}{\mathcal{I}\left( {i,r,j} \right)}}}$${ɛ\left( {M,\Diamond_{R},\Phi} \right)\underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i \in {ɛ{({M,\Phi})}}}{\bigcup\limits_{r \in R}{{\left( {r,i} \right)}{\quad {{where},{{\langle_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l} \right\rbrack}_{\zeta} \right\rbrack_{\beta}\rangle}\underset{\underset{\_}{\_}}{\Delta}\left\{ \begin{matrix}{\quad \left\{ {}_{\alpha}\left\lbrack {}_{\gamma^{\prime}}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta^{\prime},\varepsilon^{\prime}}\left\lbrack {k^{\prime},l} \right\rbrack}_{\zeta^{\prime}} \right\rbrack_{\beta} \right\}} \\{\quad {{{where}\quad j^{\prime}} = {\min \left( {j,l} \right)}}} \\{\quad {k^{\prime} = {\max \left( {k,i} \right)}}} \\{\quad {\gamma^{\prime} = {{\gamma\bigwedge i} \neq {- \infty}}}} \\{\quad {\delta^{\prime} = {{\delta\bigwedge{\min \left( {j,l} \right)}} \neq {\infty\bigwedge\left( {j < {l\bigvee{\zeta\bigwedge\alpha\bigwedge\beta}}} \right)}}}} \\{\quad {\varepsilon^{\prime} = {{\varepsilon\bigwedge{\max \left( {k,i} \right)}} \neq {{- \infty}\bigwedge\left( {k > {i\bigvee{\gamma\bigwedge\beta\bigwedge\alpha}}} \right)}}}} \\{\quad {\zeta^{\prime} = {{\zeta\bigwedge l} \neq \infty}}} \\{\quad {{{when}\quad i} \leq {j^{\prime}\bigwedge k^{\prime}} \leq {l\bigwedge}}} \\{\quad {\left\lbrack {i = \left. j^{\prime}\rightarrow\left( {\gamma^{\prime}\bigwedge\delta^{\prime}} \right) \right.} \right\rbrack\bigwedge\left\lbrack {k^{\prime} = \left. l\rightarrow\left( {\varepsilon^{\prime}\bigwedge\zeta^{\prime}} \right) \right.} \right\rbrack\bigwedge}} \\{\quad {\left\lbrack {i = \left. l\rightarrow\left( {\alpha\bigwedge\beta} \right) \right.} \right\rbrack\bigwedge}} \\{\quad {i \neq {\infty\bigwedge j^{\prime}} \neq {{- \infty}\bigwedge k^{\prime}} \neq {\infty\bigwedge l} \neq {- \infty}}} \\{\quad {\left\{ \quad \right\} \quad {otherwise}}}\end{matrix} \right.}}}}}$

_(α) ₁ [_(γ) ₁ [i₁, j₁]_(δ) ₁ , _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁ ∩_(α) ₂[_(γ) ₂ [i₂, j₂]_(δ) ₂ _(ε) ₂ [k₂, l₂]_(ζ) ₂ ]_(β) ₂ Δ <_(α) ₁ [_(γ)[max(I₁, i₂), min (j₁, j₂)]_(δ), _(ε)[max (k₁,k₂), min (l₁, l₂)]_(ζ)]_(β)₁ > $\begin{matrix}{{{where}\quad \gamma} = \quad \left\{ \begin{matrix}{\quad \gamma_{1}} & {\quad {i_{1} > i_{2}}} \\{\quad {\gamma_{1}\bigwedge\gamma_{2}}} & {\quad {i_{1} = i_{2}}} \\{\quad \gamma_{2}} & {\quad {i_{1} < i_{2}}}\end{matrix} \right.} \\{\delta = \quad \left\{ \begin{matrix}{\quad \delta_{1}} & {\quad {j_{1} < j_{2}}} \\{\quad {\delta_{1}\bigwedge\delta_{2}}} & {\quad {j_{1} = j_{2}}} \\{\quad \delta_{2}} & {\quad {j_{1} > j_{2}}}\end{matrix} \right.} \\{\in {= \quad \left\{ \begin{matrix}{\quad  \in_{1}} & {k_{1} > k_{2}} \\{\quad {\in_{1}{\bigwedge \in_{2}}}} & {k_{1} = k_{2}} \\{\quad  \in_{2}} & {k_{1} < k_{2}}\end{matrix} \right.}} \\{\zeta = \quad \left\{ \begin{matrix}{\quad \zeta_{1}} & {\quad {l_{1} < l_{2}}} \\{\quad {\zeta_{1}\bigwedge\zeta_{2}}} & {\quad {l_{1} = l_{2}}} \\{\quad \zeta_{2}} & {\quad {l_{1} > l_{2}}}\end{matrix} \right.}\end{matrix}\quad$

when α₁=α₂

β₁=β₂ { } otherwise$_{\alpha}{\left\lbrack {}_{\gamma}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l} \right\rbrack_{\zeta}} \right\rbrack_{\beta}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \begin{pmatrix}{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},k} \right\rbrack_{- \in}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{\zeta}\left\lbrack {l,\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},i} \right\rbrack_{\gamma},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{\delta}{\left\lbrack {j,\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}}\end{pmatrix}}$

SPAN (_(α) ₁ [_(γ) ₁ [i₁, j₁]_(δ) ₁ _l , _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁ ,_(α) ₂ [_(γ) ₂ [i₂, j₂]_(δ) ₂ , _(ε) ₂ [k₂, l₂]_(ζ) ₂ ]_(β) ₂ )Δ${SPAN}\quad \left( {}_{\alpha_{1}}{\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1},\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j_{2}} \right\rbrack_{\delta_{2}},_{\in_{2}}\left\lbrack {k_{2},l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack}_{\beta_{2}} \right)\quad {\underset{\underset{\_}{\_}}{\Delta}\begin{pmatrix}{\quad {{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack_{\beta_{2}}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}\bigcup}} \\{\quad {\langle_{\alpha}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack_{\beta_{2}}\rangle}}\end{pmatrix}}$

where j=min (j₁, j₂) k=max (k₁, k₂) δ=[(δ₁

j₁≦j₂)

(δ₂

j₁≧j₂)] ε=[(ε₁

k₁≧k₂)

(ε₂

k₁≦k₂)]$\quad \left( {= {,i}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}\quad \left\{ i \right\}$${\left( {< {,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2} \in {({T,F})}}}{\langle_{\alpha_{2}}\left\lbrack {}_{{\beta_{1}\bigwedge{{{\alpha_{2}\bigwedge} \in_{1}}}}}{\left\lbrack {k_{1},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}$${\left( {> {,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2} \in {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},j_{1}} \right\rbrack}_{{\alpha_{1}\bigwedge{{\beta_{2}\bigwedge\delta_{1}}}}} \right\rbrack_{\beta_{2}}\rangle}$${\left( {m,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\beta_{2} \in \quad {\{{T,F}\}}}{\langle_{\beta_{1}}\left\lbrack {}_{\in_{1}}{\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1},T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}$${\left( {{mi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,\gamma_{1}}\left\lbrack {i_{1},j_{1}} \right\rbrack}_{\delta_{1}} \right\rbrack_{\alpha_{1}}\rangle}$${\left( {o,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left\lbrack {i_{1},l_{1}} \right\rbrack_{{\beta_{1}\bigwedge\alpha_{2}\bigwedge\zeta_{1}},{{\beta_{1}\bigwedge\beta_{2}}}, \in_{1}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}$${\quad \left( {{oi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},{\alpha_{1}\bigwedge\beta_{2}\bigwedge\gamma_{1}}}\left\lbrack {i_{1},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge\beta_{2}\bigwedge\zeta_{1}} \right\rbrack_{\beta_{2}}\rangle}$${\quad \left( {s,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\beta_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},{{{\beta_{1}\bigwedge\beta_{2}\bigwedge} \in_{1}}}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}$${\quad \left( {{si},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\beta_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},T}\left\lbrack {{- \infty},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack_{\beta_{2}}\rangle}$${\quad \left( {f,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2}\varepsilon \quad {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}$${\quad \left( {{fi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left\lbrack {i_{1},\infty} \right\rbrack_{T, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}\rangle}$${\quad \left( {d,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},{{{\beta_{1}\bigwedge\beta_{2}\bigwedge} \in_{1}}}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}$${\quad \left( {{di},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{\alpha_{2},{\beta_{2} \in {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left\lbrack {i_{1},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack_{\beta_{2}}\rangle}$${and},{{\left( {i,r,j} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\bigcup\limits_{i^{\prime} \in \quad {\quad {({r^{- 1},j})}}}{\bigcup\limits_{i^{\prime} \in \quad {i^{\prime}\bigcap i}}{\bigcup\limits_{j^{\prime} \in \quad {{({r,i})}}}{\bigcup\limits_{j^{\prime} \in \quad {j^{\prime}\bigcap\quad j}}{{{SPAN}\left( {i^{\prime\prime},j^{\prime\prime}} \right)}.}}}}}$


7. A program storage device readable by machine, tangibly embodying aprogram of instructions executable by machine to perform method stepsfor computing all occurrences of a compound event from occurrences ofprimitive events where the compound event is a defined combination ofthe primitive events, the method comprising the steps of: (a) definingprimitive event types; (b) defining combinations of the primitive eventtypes as a compound event type; (c) inputting the primitive eventoccurrences, such occurrences being specified as the set of temporalintervals over which a given primitive event type is true; and (d)computing the compound event occurrences, such occurrences beingspecified as the set of temporal intervals over which the compound eventtype is true, wherein the sets of temporal intervals in steps (c) and(d) are specified as smaller sets of spanning intervals, each spanninginterval representing a set of intervals.
 8. The program storage deviceaccording to claim 7, wherein the spanning intervals take the form _(α[)_(γ)[i, j]_(δ) _(ε)[k, l]_(ζ)]_(β), where α, β, γ, δ, ε, and ζ areBoolean values, i, j, k, and l are real numbers, _(α[) _(γ)[i, j, ]_(δ),_(ε)[k, l]_(ζ)]_(β) represents the set of all intervals _(α[p, q]) _(β)where i≦_(γ)p≦_(δ)j and k≦_(ε)q≦_(ζ)l, _(α)[p, q]_(β) represents the setof all points r where p≦_(α)r≦_(β)q, and x≦_(θ)y means x<y when θ istrue and x<y when θ is false.
 9. The program storage device according toclaim 8, wherein the compound event type in step (b) is specified as anexpression in temporal logic.
 10. The program storage device accordingto claim 9, wherein the temporal logic expressions are constructed usingthe logical connectives ∀, ∃,

,

_(R), ⋄_(R), and

, where R ranges over sets of relations between one-dimensionalintervals.
 11. The program storage device according to claim 10, whereinthe relations are =, <,>, m, mi, o, oi, s, si, f, fi, d, and di.
 12. Themethod according to claim 11, wherein the compound event occurrences arecomputed using the following set of equations: $\begin{matrix}{ɛ\left( {M,{p\left( {c_{1},\ldots \quad,c_{n}} \right)}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & \left\{ i \middle| {{{p\left( {c_{1},\ldots \quad,c_{n}} \right)}@i}\quad \varepsilon \quad M} \right\} \\{ɛ\left( {M,{\Phi\bigvee\Psi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {{ɛ\left( {M,\Phi} \right)}\bigcup{ɛ\left( {M,\Psi} \right)}} \\{ɛ\left( {M,{\forall{x\quad \Phi}}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {{\bigcup\limits_{i_{1}\varepsilon \quad {ɛ{({M,{\Phi {\lbrack{x:=c_{1}}\rbrack}}})}}}\ldots}\bigcup\limits_{i_{n}\varepsilon \quad {ɛ{({M,{\Phi {\lbrack{x:=c_{n}}\rbrack}}})}}}{i_{1}\bigcap\ldots\bigcap i_{n}}} \\\quad & \quad & {{{where}\quad {C(M)}} = \left\{ {c_{1},\ldots \quad,c_{n}} \right\}} \\{ɛ\left( {M,{\exists{x\quad \Phi}}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {\bigcup\limits_{c\quad \varepsilon \quad {C{(M)}}}{ɛ\left( {M,{\Phi \left\lbrack {x:=c} \right\rbrack}} \right)}} \\{ɛ\left( {M,{\Phi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {{\bigcup\limits_{i_{1}^{\prime}\varepsilon {i_{1}}}\ldots}\bigcup\limits_{i_{n}^{\prime}\varepsilon {\quad i_{n}}}{i_{1}^{\prime}\bigcap\ldots\bigcap i_{n}^{\prime}}} \\\quad & \quad & {{{where}\quad {ɛ\left( {M,\Phi} \right)}} = \left\{ {i_{1},\ldots \quad,i_{n}} \right\}} \\{ɛ\left( {M,{\Phi \bigwedge_{R}\Psi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {\bigcup\limits_{i\quad \varepsilon \quad {ɛ{({M,\Phi})}}}{\bigcup\limits_{j\quad \varepsilon \quad {ɛ{({M,\Psi})}}}{\bigcup\limits_{r\quad \varepsilon \quad R}{\left( {i,r,j} \right)}}}} \\{ɛ\left( {M,{\Diamond_{R}\Phi}} \right)} & \underset{\underset{\_}{\_}}{\Delta} & {\bigcup\limits_{i\quad \varepsilon \quad {ɛ{({M,\Phi})}}}{\bigcup\limits_{r\quad \varepsilon \quad R}{\left( {r,i} \right)}}} \\{{where},} & \quad & \quad\end{matrix}$${\langle_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l} \right\rbrack}_{\zeta} \right\rbrack_{\beta}\rangle}\underset{\underset{\_}{\_}}{\Delta}\left\{ \begin{matrix}\left\{ {}_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k^{\prime},l} \right\rbrack}_{\zeta} \right\rbrack_{\beta} \right\} \\\begin{matrix}{where} & {j^{\prime} = {\min \left( {j,l} \right)}} \\\quad & {k^{\prime} = {\max \left( {k,i} \right)}} \\\quad & {\gamma^{\prime} = {{\gamma\bigwedge i} \neq {- \infty}}} \\\quad & {\delta^{\prime} = {{\delta\bigwedge{\min \left( {j,l} \right)}} \neq {\infty\bigwedge\left( {j < {l\bigvee{\zeta\bigwedge\alpha\bigwedge\beta}}} \right)}}} \\\quad & {\varepsilon^{\prime} = {{\varepsilon\bigwedge{\max \left( {k,i} \right)}} \neq {{- \infty}\bigwedge\left( {k > {i\bigvee{\gamma\bigwedge\beta\bigwedge\alpha}}} \right)}}} \\\quad & {\zeta^{\prime} = {{\zeta\bigwedge l} \neq \infty}} \\{when} & {i \leq {j^{\prime}\bigwedge k^{\prime}} \leq {l\bigwedge}} \\\quad & {\left\lbrack {i = \left. j^{\prime}\rightarrow\left( {\gamma^{\prime}\bigwedge\delta^{\prime}} \right) \right.} \right\rbrack\bigwedge\left\lbrack {k^{\prime} = \left. l\rightarrow\left( {\varepsilon^{\prime}\bigwedge\zeta^{\prime}} \right) \right.} \right\rbrack\bigwedge} \\\quad & {\left\lbrack {i = \left. l\rightarrow\left( {\alpha\bigwedge\beta} \right) \right.} \right\rbrack\bigwedge} \\\quad & {i \neq {\infty\bigwedge j^{\prime}} \neq {{- \infty}\bigwedge k^{\prime}} \neq {\infty\bigwedge l} \neq {- \infty}} \\\left\{ \quad \right\} & {otherwise}\end{matrix}\end{matrix} \right.$

_(α) ₁ [_(γ) ₁ [i₁j₁]_(δ) ₁ , _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁ ∩_(α) ₂[_(γ) ₂ [i₂, j₂]_(δ) ₂ , _(ε) ₂ [k₂, i₂]_(ζ) ₂ ]_(β) ₂ Δ <_(α) ₁[_(γ)[max (i₁, i₂), min (j₁, j₂)_(δ), _(ε)[max (k₁, k₂), min (l₁,l₂)]_(ζ)]_(β) ₁ > $\begin{matrix}{{{where}\quad \gamma} = \left\{ \begin{matrix}{\quad \gamma_{1}} & {\quad {i_{1} > i_{2}}} \\{\quad {\gamma_{1}\bigwedge\gamma_{2}}} & {\quad {i_{1} = i_{2}}} \\{\quad \gamma_{2}} & {\quad {i_{1} < i_{2}}}\end{matrix} \right.} \\{\delta = \left\{ {{\begin{matrix}{\quad \delta_{1}} & {\quad {j_{1} < j_{2}}} \\{\quad {\delta_{1}\bigwedge\delta_{2}}} & {\quad {j_{1} = j_{2}}} \\{\quad \delta_{2}} & {\quad {j_{1} > j_{2}}}\end{matrix} \in} = \left\{ \begin{matrix}{\quad  \in_{1}} & {k_{1} > k_{2}} \\{\quad {\in_{1}{\bigwedge\varepsilon_{2}}}} & {k_{1} = k_{2}} \\{\quad  \in_{2}} & {k_{1} < k_{2}}\end{matrix} \right.} \right.} \\{\zeta = \left\{ \begin{matrix}{\quad \zeta_{1}} & {\quad {l_{1} < l_{2}}} \\{\quad {\zeta_{1}\bigwedge\zeta_{2}}} & {\quad {l_{1} = l_{2}}} \\{\quad \zeta_{2}} & {\quad {l_{1} > l_{2}}}\end{matrix} \right.}\end{matrix}$

when α₁=α₂

β₁=β₂ { } otherwise$\quad_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l} \right\rbrack_{\zeta}} \right\rbrack_{\beta}\underset{\underset{\_}{\_}}{\Delta}\quad \begin{pmatrix}{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},k} \right\rbrack_{- \in}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{\zeta}\left\lbrack {l,\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},i} \right\rbrack_{\gamma},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{\delta}{\left\lbrack {j,\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta}\rangle}}\end{pmatrix}$

SPAN (_(α) ₁ [_(γ) ₁ [i₁, j₁]_(δ) ₁ , _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁ ,_(α) ₂ [_(γ) ₂ [i₂, j₂]_(δ) ₂ , _(ε) ₂ [k₂, l₂ _(ζ) ₂ ]_(β) ₂ )Δ$\quad\quad \begin{pmatrix}{{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}\bigcup} \\{{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack_{\beta_{2}}\rangle}\bigcup} \\{{\langle_{\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}\bigcup} \\{{\langle_{\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta},_{\in}\left\lbrack {k,l_{2}} \right\rbrack_{\zeta_{2}}} \right\rbrack_{\beta_{2}}\rangle}\quad}\end{pmatrix}$

where j=min (j₁, j₂) k=max (k₁, k₂) δ=[(δ₁

j₁≦j₂)

(δ₂

j₁≧j₂)] ε=[(ε₁

k₁≧k₂)

(ε₂

k₁≦k₂)] $\begin{matrix}{{\left( {= {,i}} \right)}\overset{\Delta}{=}\quad \left\{ i \right\}} \\{{\left( {< {,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}}} \right)}\overset{\Delta}{=}\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{{\beta_{1}\bigwedge{{\alpha_{2}\bigwedge\varepsilon_{1}}}}}{\left\lbrack {k_{1},\infty} \right\rbrack_{T},_{T}\left. \left\lbrack {{- \infty},\infty} \right\rbrack_{T} \right\rbrack_{\beta_{2}}}\rangle \right.}}} \\{{\left( {> {,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}}} \right)}\overset{\Delta}{=}\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{T}\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{\alpha_{1}\bigwedge{{\beta_{2}\bigwedge\delta_{1}}}}}} \right\rbrack_{\beta_{2}}\rangle}}} \\{{\left( {m,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)}\overset{\Delta}{=}\quad {\bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\beta_{1}}\left\lbrack {}_{\varepsilon_{1}}{\left\lbrack {k_{1},l_{1}} \right\rbrack_{\zeta_{1}},_{T}\left\lbrack {{- \infty},\infty} \right\rbrack_{T}} \right\rbrack_{\beta_{2}}\rangle}}} \\{{\left( {{m\quad i},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}},_{\varepsilon_{1}}\left\lbrack {k_{1},\left. l_{1} \right\rbrack} \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)}\overset{\Delta}{=}\quad {\bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T},_{\gamma_{1}}\left. \left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}} \right\rbrack_{\alpha_{1}}}\rangle \right.}}}\end{matrix}$

${{\left( {o,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\quad  \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle{\alpha_{2}\left\lbrack {\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge{{\gamma_{1}\left\lbrack {i_{1},l_{1}} \right\rbrack}_{{\beta_{1}\bigwedge\alpha_{2}\bigwedge\zeta_{1}},{{{\beta_{1}\bigwedge\beta_{2}\bigwedge} \in_{1}}}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T}}}} \right\rbrack}_{\beta_{2}}\rangle}$${{\left( {{oi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle{\alpha_{2}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},{\alpha_{1}\bigwedge\beta_{2}\bigwedge\gamma_{1}}}\left\lbrack {i_{1},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack}_{\beta_{2}}\rangle}$${{\left( {s,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\beta_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},{{{\beta_{1}\bigwedge\beta_{2}\bigwedge} \in_{1}}}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T} \right\rbrack_{\beta_{2}}\rangle}$${{\left( {{si},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\beta_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},T}\left\lbrack {{- \infty},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack_{\beta_{2}}\rangle}$${{\left( {f,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{{\alpha_{1}\bigwedge\alpha_{2}}\delta_{1}}}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}\rangle}$${{\left( {{fi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2} \in \quad {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left\lbrack {i_{1},\infty} \right\rbrack_{T, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}\rangle}$${{\left( {d,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle_{\alpha_{2}}{\left\lbrack {}_{T}{\left\lbrack {{- \infty},j_{1}} \right\rbrack_{{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},{{{\beta_{1}\bigwedge\beta_{2}\bigwedge} \in_{1}}}}\left\lbrack {k_{1},\infty} \right\rbrack}_{T} \right\rbrack \beta_{2}}\rangle}$${{\left( {{di},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1}, \in_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2} \in \quad {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left\lbrack {i_{1},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},l_{1}} \right\rbrack}_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}} \right\rbrack_{\beta_{2}}\rangle}$${and},\quad {{{\left( {i,r,j} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{i^{\prime} \in \quad {\quad {({r^{- 1},j})}}}{\bigcup\limits_{i^{\prime} \in \quad {i^{\prime}\bigcap\quad i}}{\bigcup\limits_{j^{\prime} \in \quad {\quad {({r,i})}}}{\bigcup\limits_{j^{\prime} \in \quad {j^{\prime}\bigcap\quad j}}{{SPAN}\quad {\left( {i^{\prime\prime},j^{\prime\prime}} \right).}}}}}}$


13. A computer program product embodied in a computer-readable mediumfor implementing the computation of all occurrences of a compound eventfrom occurrences of primitive events where the compound event is adefined combination of the primitive events, the computer programproduct comprising: (a) computer readable program code means fordefining primitive event types; (b) computer readable program code meansfor defining combinations of the primitive event types as a compoundevent type; (c) computer readable program code means for inputting theprimitive event occurrences, such occurrences being specified as the setof temporal intervals over which a given primitive event type is true;and (d) computer readable program code means for computing the compoundevent occurrences, such occurrences being specified as the set oftemporal intervals over which the compound event type is true, whereinthe sets of temporal intervals in steps (c) and (d) are specified assmaller sets of spanning intervals, each spanning interval representinga set of intervals.
 14. The computer program product according to claim13, wherein the spanning intervals take the form _(α)[_(γ)[i, j]_(δ),_(ε)[k, l₁]_(ζ)]_(β), where α, β, γ, δ, ε, and ζ are Boolean values, i,j, k, and l are real numbers, _(α)[_(γ)[i, j]_(δ), _(ε)[k, l]_(ζ)]_(β)represents the set of all intervals _(α)[p, q]_(β) where i≦_(γ)p≦_(δ)jand k≦_(ε)q≦_(ζ)l, _(α)[p, q]_(β) represents the set of all points rwhere p≦_(α)r≦_(β)q, and x≦_(θ)y means x≦y when θ is true and x<y when θis false.
 15. The computer program product according to claim 14,wherein the compound event type in (b) is specified as an expression intemporal logic.
 16. The computer program product according to claim 15,wherein the temporal logic expressions are constructed using the logicalconnectives ∀, ∃,

,

_(R), ⋄_(R), and

, where R ranges over sets of relations between one-dimensionalintervals.
 17. The computer program product according to claim 16,wherein the relations are =, <, >, m, mi, o, oi, s, si, f, fi, d, anddi.
 18. The method according to claim 17, wherein the compound eventoccurrences are computed using the following set of equations:$\begin{matrix}{{ɛ\left( {M,{p\left( {c_{1},\ldots \quad,c_{n}} \right)}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \left\{ \left. {i{{{{p\left( {c_{1},\ldots \quad,c_{n}} \right)}\quad@\quad i} \in M}}} \right\} \right.} \\{{{ɛ\left( {M,{\Phi\bigvee\Psi}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}\quad {ɛ\left( {M,\Phi} \right)}}\bigcup{ɛ\quad \left( {M,\Psi} \right)}} \\{{{ɛ\left( {M,{\forall{x\quad \Phi}}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{i_{1} \in \quad {ɛ\quad {({M,{\Phi \quad\lbrack{x:=c_{1}}\rbrack}})}}}\ldots \bigcup\limits_{i_{n} \in \quad {ɛ\quad {({M,{\Phi {\lbrack{x:=c_{n}}\rbrack}}})}}}{i_{1}\bigcap\quad \ldots\bigcap i_{n}}} \\{\quad {{{where}\quad {C(M)}} = \left\{ {c_{1},\ldots,c_{n}} \right\}}} \\{{ɛ\quad \left( {M,{\exists{x\quad \Phi}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{c\quad \in \quad {C{(M)}}}{ɛ\left( {M,{\Phi \left\lbrack {x:=c} \right\rbrack}} \right)}} \\{{ɛ\quad \left( {M,{\Phi}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{i_{1}^{\prime} \in {i_{1}}}\ldots \bigcup\limits_{i_{n}^{\prime} \in {i_{n}}}{i_{1}^{\prime}\bigcap\quad \ldots \quad\bigcap i_{n}^{\prime}}} \\{\quad {{{where}\quad ɛ\quad \left( {M,\Phi} \right)} = \left\{ {i_{1},\ldots \quad,i_{n}} \right\}}} \\{{ɛ\quad \left( {M,{\Phi \bigwedge_{R}\Psi}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{i\quad \in \quad {ɛ\quad {({M,\Phi})}}}{\bigcup\limits_{j \in \quad {ɛ\quad {({M,\Psi})}}}{\bigcup\limits_{r\quad \in \quad R}{\left( {i,r,j} \right)}}}} \\{{ɛ\quad \left( {M,{♦_{R}\Phi}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{{i\quad \in \quad {ɛ\quad {({M,\Phi})}}}\quad}{\bigcup\limits_{r\quad \in \quad R}{\left( {r,i} \right)}}}\end{matrix}$${where},{{\langle_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l} \right\rbrack}_{\zeta} \right\rbrack_{\beta}\rangle}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \left\{ \begin{matrix}{\quad \left\{ {}_{\alpha}\left\lbrack {}_{\gamma}{\left\lbrack {i,j^{\prime}} \right\rbrack_{\delta^{\prime},\varepsilon^{\prime}}\left\lbrack {k^{\prime},l} \right\rbrack}_{\zeta^{\prime}} \right\rbrack_{\beta} \right\} \quad} \\\begin{matrix}{{{where}\quad j^{\prime}} = \quad {\min \quad \left( {j,l} \right)}} \\{k^{\prime} = \quad {\max \left( {k,i} \right)}} \\{\gamma^{\prime} = \quad {{\gamma\bigwedge i} \neq {- \infty}}} \\{\delta^{\prime} = \quad {{{\delta\bigwedge\min}\quad \left( {j,l} \right)} \neq {\infty\bigwedge\left( {j < {l\bigvee{\zeta\bigwedge\alpha\bigwedge\beta}}} \right)}}} \\{\varepsilon^{\prime} = \quad {{{\varepsilon\bigwedge\max}\quad \left( {k,i} \right)} \neq {{- \infty}\bigwedge}}} \\{\quad \left( {k > {i\bigvee{\gamma\bigwedge\beta\bigwedge\alpha}}} \right)} \\{\zeta^{\prime} = \quad {{\zeta\bigwedge l} \neq \infty}} \\{{{when}\quad i} \leq \quad {j^{\prime}\bigwedge k^{\prime}} \leq {l\bigwedge}} \\{\left\lbrack {i = \quad {j^{\prime}->\left( {\gamma^{\prime}\bigwedge\delta^{\prime}} \right)}} \right\rbrack\bigwedge\left\lbrack {k^{\prime} = {l->\left( {\varepsilon^{\prime}\bigwedge\zeta^{\prime}} \right)}} \right\rbrack\bigwedge} \\{\left\lbrack {i = \quad {l->\left( {\alpha\bigwedge\beta} \right)}} \right\rbrack\bigwedge} \\{i \neq \quad {\infty\bigwedge j^{\prime}} \neq {{- \infty}\bigwedge k^{\prime}} \neq {\infty\bigwedge l} \neq {- \infty}} \\{{\{\}}\quad {otherwise}}\end{matrix}\end{matrix} \right.}$

_(α) ₁ [_(γ) ₁ [i₁, j₁]_(δ) ₁ , _(ε) ₁ [k₁, l₁]_(ζ) ₁ ]_(β) ₁ ∩_(α) ₂[_(γ) ₂ [i₂, j₂]_(δ) ₂ , _(ε) ₂ [k₂, l₂]_(ζ) ₂ ]_(β) ₂ Δ <_(α) ₁[_(γ)[max (i₁, i₂), min (j₁, j₂)]_(δ), _(ε)[max (k₁, k₂), min (l₁,l₂)]_(ζ)]_(β) ₁ > $\begin{matrix}{{{where}\quad \gamma} = \left\{ \begin{matrix}{\quad \gamma_{1}} & {\quad {i_{1} > i_{2}}} \\{\quad {\gamma_{1}\bigwedge\gamma_{2}}} & {\quad {i_{1} = i_{2}}} \\{\quad \gamma_{2}} & {\quad {i_{1} < i_{2}}}\end{matrix} \right.} \\{\delta = \left\{ {{\begin{matrix}{\quad \delta_{1}} & {\quad {j_{1} < j_{2}}} \\{\quad {\delta_{1}\bigwedge\delta_{2}}} & {\quad {j_{1} = j_{2}}} \\{\quad \delta_{2}} & {\quad {j_{1} > j_{2}}}\end{matrix} \in} = \left\{ \begin{matrix}{\quad  \in_{1}} & {k_{1} > k_{2}} \\{\quad {\in_{1}{\bigwedge\varepsilon_{2}}}} & {k_{1} = k_{2}} \\{\quad  \in_{2}} & {k_{1} < k_{2}}\end{matrix} \right.} \right.} \\{\zeta = \left\{ \begin{matrix}{\quad \zeta_{1}} & {\quad {l_{1} < l_{2}}} \\{\quad {\zeta_{1}\bigwedge\zeta_{2}}} & {\quad {l_{1} = l_{2}}} \\{\quad \zeta_{2}} & {\quad {l_{1} > l_{2}}}\end{matrix} \right.}\end{matrix}$

when α₁=α₂

β₁=β₂ { } otherwise$_{\alpha}{\left\lbrack {}_{\gamma}{\left\lbrack {i,j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l} \right\rbrack}_{\zeta} \right\rbrack_{\beta}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \begin{pmatrix}{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},k} \right\rbrack}_{\varepsilon} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,{\zeta}}\left\lbrack {l,\infty} \right\rbrack}_{T} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},i} \right\rbrack_{{\gamma},T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{\delta}{\left\lbrack {j,\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta}\rangle}\bigcup}} \\{\quad {\langle_{\alpha}\left\lbrack {}_{T}{\left\lbrack {{- \infty},\infty} \right\rbrack_{T,T}\left\lbrack {{- \infty},\infty} \right\rbrack}_{T} \right\rbrack_{\beta}\rangle}}\end{pmatrix}}$${{SPAN}\left( {}_{\alpha_{1}}{\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j_{1}} \right\rbrack_{\delta_{1},\varepsilon_{1}}\left\lbrack {k_{1},l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1},\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j_{2}} \right\rbrack_{\delta_{2},\varepsilon_{2}}\left\lbrack {k_{2},l_{2}} \right\rbrack}_{\zeta_{2}} \right\rbrack}_{\beta_{2}} \right)}\quad {\underset{\underset{\_}{\_}}{\Delta}\begin{pmatrix}{\quad {{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left\lbrack {i_{1},j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l_{2}} \right\rbrack}_{\zeta_{2}} \right\rbrack_{\beta_{2}}\rangle}\bigcup}} \\{\quad {{\langle_{\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l_{1}} \right\rbrack}_{\zeta_{1}} \right\rbrack_{\beta_{1}}\rangle}\bigcup}} \\{\quad {\langle_{\alpha_{2}}\left\lbrack {}_{\gamma_{2}}{\left\lbrack {i_{2},j} \right\rbrack_{\delta,\varepsilon}\left\lbrack {k,l_{2}} \right\rbrack}_{\zeta_{2}} \right\rbrack_{\beta_{2}}\rangle}}\end{pmatrix}}$

where j=min (j₁, j₂) k=max (k₁, k₂) δ=[(δ₁

j₁≦j₂)

(δ₂

j₁≧j₂)] ε=[(ε₁

k₁≧k₂)

(ε₂

k₁≦k₂)] $\begin{matrix}{{\left( {= {,i}} \right)}\quad \underset{\underset{\_}{\_}}{\Delta}\quad \left\{ i \right\}} \\{{\left( {< {,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{{- \beta_{1}}\bigwedge{{\alpha_{2}\bigwedge\varepsilon_{1}}}}{\left( \left\lbrack {k_{1},\infty} \right\rbrack \right)_{T},_{T}\left( \left\lbrack {{- \infty},\infty} \right\rbrack \right)_{T}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {< {,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left( \left\lbrack {{- \infty},\infty} \right\rbrack \right)_{T},_{T}\left( \left\lbrack {{- \infty},j_{1}} \right\rbrack \right)_{{\alpha_{1}\bigwedge{{\beta_{2}\bigwedge\delta_{1}}}}}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {m,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\beta_{1}}\left\lbrack {}_{\varepsilon_{1}}{\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}},_{T}\left( \left\lbrack {{- \infty},\infty} \right\rbrack \right)_{T}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {{m\quad i},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left( \left\lbrack {{- \infty},\infty} \right\rbrack \right)_{T},_{\gamma_{1}}\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}}} \right\rbrack_{\alpha_{1}}\rangle}} \\{{\left( {o,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad {\bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left\lbrack {i_{1},l_{1}} \right\rbrack_{\beta_{1}\bigwedge\alpha_{2}\bigwedge\zeta_{1}},_{{\beta_{1}\bigwedge\beta_{2}\bigwedge\varepsilon_{1}}}\left\lbrack {k_{1},\infty} \right\rbrack_{T}} \right\rbrack_{\beta_{2}}\rangle}}} \\{{\left( {{oi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left( \left\lbrack {{- \infty},j_{1}} \right\rbrack \right)_{{\alpha_{1}\bigwedge\alpha_{2}\bigwedge\delta_{1}}},_{\alpha_{1}\bigwedge\beta_{2}\bigwedge\gamma_{1}}\left( \left\lbrack {i_{1},l_{1}} \right\rbrack \right)_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {s,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{{\beta_{1}\bigwedge\beta_{2}\bigwedge\varepsilon_{1}}}\left( \left\lbrack {k_{1},\infty} \right\rbrack \right)_{T}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {{si},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\beta_{2}\varepsilon {\{{T,F}\}}}{\langle_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{T}\left( \left\lbrack {{- \infty},l_{1}} \right\rbrack \right)_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {f,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left( \left\lbrack {{- \infty},j_{1}} \right\rbrack \right)_{{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\delta_{1}}}}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}} \\{{\left( {{fi},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2}\varepsilon {\{{T,F}\}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left( \left\lbrack {i_{1},\infty} \right\rbrack \right)_{T},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}\rangle}} \\{{\left( {d,_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{T}{\left( \left\lbrack {{- \infty},j_{1}} \right\rbrack \right)_{{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\delta_{1}}}}}{,_{{\beta_{1}\bigwedge\beta_{2}\bigwedge}}}_{\varepsilon_{1}}\left( \left\lbrack {k_{1},\infty} \right\rbrack \right)_{T}} \right\rbrack_{\beta_{2}}\rangle}} \\{{\left( {{di},_{\alpha_{1}}\left\lbrack {}_{\gamma_{1}}{\left( \left\lbrack {i_{1},j_{1}} \right\rbrack \right)_{\delta_{1}},_{\varepsilon_{1}}\left( \left\lbrack {k_{1},l_{1}} \right\rbrack \right)_{\zeta_{1}}} \right\rbrack_{\beta_{1}}} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad \bigcup\limits_{\alpha_{2},{\beta_{2}\varepsilon {\{{T,F}\}}}}{\langle_{\alpha_{2}}\left\lbrack {}_{\alpha_{1}\bigwedge{{\alpha_{2}\bigwedge\gamma_{1}}}}{\left( \left\lbrack {i_{1},\infty} \right\rbrack \right)_{T},_{T}\left( \left\lbrack {{- \infty},l_{1}} \right\rbrack \right)_{\beta_{1}\bigwedge{{\beta_{2}\bigwedge\zeta_{1}}}}} \right\rbrack_{\beta_{2}}\rangle}} \\{{and},\quad {{\quad \left( {i,r,j} \right)\quad \underset{\underset{\_}{\_}}{\Delta}}\quad {\bigcup\limits_{i^{\prime}{{\varepsilon }{({r^{- 1},j})}}}{\bigcup\limits_{{i^{\prime}\varepsilon \quad i^{\prime}}\bigcap i}{\bigcup\limits_{j^{\prime}{{\varepsilon }{({r,i})}}}{\bigcup\limits_{{j^{\prime}\varepsilon \quad j^{\prime}}\bigcap j}{{{S{PAN}}\left( {i^{''},j^{''}} \right)}.}}}}}}}\end{matrix}$